-- Logs begin at Mon 2022-08-08 01:09:49 UTC, end at Mon 2022-08-08 02:46:28 UTC. -- Aug 08 02:04:26.741403 dv-cinder-740146-18-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 02:04:26.750221 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 02:04:26 2022] *** Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: nodename: dv-cinder-740146-18-iscsi Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: machine: x86_64 Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: clock source: unix Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: pcre jit disabled Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: detected number of CPU cores: 4 Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: current working directory: / Aug 08 02:04:26.752241 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: detected binary path: /usr/bin/uwsgi-core Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: your processes number limit is 47680 Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: your memory page size is 4096 bytes Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: detected max file descriptor number: 2048 Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: lock engine: pthread robust mutexes Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: thunder lock: enabled Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 02:04:26.755816 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 02:04:26.792905 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: Python main interpreter initialized at 0x55ab642a59c0 Aug 08 02:04:26.792905 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: python threads support enabled Aug 08 02:04:26.792905 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: your server socket listen backlog is limited to 100 connections Aug 08 02:04:26.792905 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: your mercy for graceful operations on workers is 90 seconds Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: *** Operational MODE: preforking *** Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: *** uWSGI is running in multiple interpreter mode *** Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: spawned uWSGI master process (pid: 80527) Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: spawned uWSGI worker 1 (pid: 80537, cores: 1) Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: spawned uWSGI worker 2 (pid: 80538, cores: 1) Aug 08 02:04:26.795973 dv-cinder-740146-18-iscsi devstack@keystone.service[80527]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 02:04:26.793872 dv-cinder-740146-18-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 02:04:32.294207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:04:32.294207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: from cryptography.utils import int_from_bytes Aug 08 02:04:32.295017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:04:32.295017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: from cryptography.utils import int_from_bytes Aug 08 02:04:32.295146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:04:32.295146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: from cryptography.utils import int_from_bytes Aug 08 02:04:32.295146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 02:04:32.295146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: from cryptography.utils import int_from_bytes Aug 08 02:04:32.345917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:04:32.346114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:04:32.346312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] command line args: [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:04:32.346590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:04:32.346772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ================================================================================ {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:04:32.347044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] admin_token = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.347297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] config_dir = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.347492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.347678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] config_source = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.347864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] debug = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.348068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_config_append = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.349881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_dir = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.350064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_file = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.350305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.350558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.350816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.351069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.351254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.351437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.351687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.351938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.352179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.352419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.352656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.352919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.353171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.353482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.353700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.354199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.354385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] publish_errors = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.354625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.354861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.355043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.355293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.355660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.355912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] standard_threads = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.356151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.356399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.356581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.356785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] use_journal = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.356967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] use_json = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.357147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] use_stderr = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.357328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] use_syslog = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.357567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.357828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.358073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.358338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.358592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.358846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.359092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.359347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.359590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.359843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.360097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.360353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.360593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.360878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.361121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.361375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.361630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.361873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.362132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.362393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.362647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.362888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.363148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.363429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.363690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.363948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.364187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.364424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.364681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.364922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.365177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.365432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.365669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.365907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.366161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.366415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.366659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.366829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 02:04:32.366918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.367046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 02:04:32.367125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.367266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] command line args: [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 02:04:32.367368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.367592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 02:04:32.367672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.367791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ================================================================================ {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 02:04:32.367871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.368060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.368144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] admin_token = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.368299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.368441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] config_dir = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.368525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.368699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.368864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.368940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] config_source = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.369036 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.369113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] debug = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.369275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.369365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.369519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.369597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.369755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.369833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.369996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.370134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.370239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.370479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.370556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.370719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.370805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.370965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.371622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_config_append = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.371738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.372179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.372179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_dir = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.372309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.372369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_file = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.372432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.372432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.372540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.372645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.372818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.372904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.373150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.373246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.373395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.373478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.373635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.373714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.373815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.373899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.374133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.374217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.374380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.374461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.374602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.374734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.374843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.375043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.375128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.375350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.375426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.375603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.375680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.375854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.375981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.376095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.376322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.376400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.376563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.376729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.376823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.377067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.377310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.377504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.377590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.377706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] publish_errors = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.377785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.378013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.378090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.378253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.378356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.378495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.378604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.378734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.378842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.379007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.379117 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.379252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.379431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] standard_threads = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.379563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.379728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.379813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.379976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.380057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.380220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.380307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.380447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] use_journal = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.380532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.380655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] use_json = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.380758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.380875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] use_stderr = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.380961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] use_syslog = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.381205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 02:04:32.381455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.381996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.382815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.external = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.383986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.384831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.password = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] auth.token = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.385886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.386628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.387953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.388921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.389976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.390938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.391914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.392908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.393986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.394945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.395979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.396955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.397904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.398881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.399843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.400982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.401937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.402953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.403937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.404914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.405996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.406934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.407922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.external = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.408971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.password = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] auth.token = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.409918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.410886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.411991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.412976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.413949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.414930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.415916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.416922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.417884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] role.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] role.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.418929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.419900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.420885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.421976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.422974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.423998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.424870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.425996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.426955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.427983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.428979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.429937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.430874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.connection = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.431850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.432982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.433074 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.433214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.433293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.433448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.434938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.435893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.436733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80538) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:04:32.436809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.437044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.437290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.437526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.437777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.438008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.438252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.438486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.438720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.438958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.439204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.439446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.439678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.439915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.440157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.440408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.440656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.440914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.441163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.441411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.441645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.441889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.442124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.442362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.442598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] role.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.442845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] role.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.443081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.443328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.443574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.443821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.444067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.444312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.444556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.444839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.445088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.445320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.445566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.445811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.446056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.446301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.446698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.446937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.447186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.447431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.447678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.447927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.448161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.448522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.448805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.449050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.449879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.450168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.450403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.450659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.450917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.451173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.451413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.451650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.451882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.452117 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.452354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.452603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.452866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.453128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.453375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.453616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.453857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.454100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.454347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.454584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.454822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.455057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.455303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.455551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.455781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.456021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.456314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.456578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.connection = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.456849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.457098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.457337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.457579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.457814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.458051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.458286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.458521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.458766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.459001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.459245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.459480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.459726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.459961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.460206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.460437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.460715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.460936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 02:04:32.461105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 02:04:32.466386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.466628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.466863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.476614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.485989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.486225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.486452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.496546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.505341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.505581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.515243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.515478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.558245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.559348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.609752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.609998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.610678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.610916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.682684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.682902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.683043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.683175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.683370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.683513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.684523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.707595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.707773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.707902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.708909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.709091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.719806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.720168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.732935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.733927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.734051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.734133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.734266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.734348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.734491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80538) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 02:04:32.751118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.751326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:04:32.751676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.751760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.751875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:04:32.751974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:04:32.752151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.752260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.752413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.752495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 02:04:32.752694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.752779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.752963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.753043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.753280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.753545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.765904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.766089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.766383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.772694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.773323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.775744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.776028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.776312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.776393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.776595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.776703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.776905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.776987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.777973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.778834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.779119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.779406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.802349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.803763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.803911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.804132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.805315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.805611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.808892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.809255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.809545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.810455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.810748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.811040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.816071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.816361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.817652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.817949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.820360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.820654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.820960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.821245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.821527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.821809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.822024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.822151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.822323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.822454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.822617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.822912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.823199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.823483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.823764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.824040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.838400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.838685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.840929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.841215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.843241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.845191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.846411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.846711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.848346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.848652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.851266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.853196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.854463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.856435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.857896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.858204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.858505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.858793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.859073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.859362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.859615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.859928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.860228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.860529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.860839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.861138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.875570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.875863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.876147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.876418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.877144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.877449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.877729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.877999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.883461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.884962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.885094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.885304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.886540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.887378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.888868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.889158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.889461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.889752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.890033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.890249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.890544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.890852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.891151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.891436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.901351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.901640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.901924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.902206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.903158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.903394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.903532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.903686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.903815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.903978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.904268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.904559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.904869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.905154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.920813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.922226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.923768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.926718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.927131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.928295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.929826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.930114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.930409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.931724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.935424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.936830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.936967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.937266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.937560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.938435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.941357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.941655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.941953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.942235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.947446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.950804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.950944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.951091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.951366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.953886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.954189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.954509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.954783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.958221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.958509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.958795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.959077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.959351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.959624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.959900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.960181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.960563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.960871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.961149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.961414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.961691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.961963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.962891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.963193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.963487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.970966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80538) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 02:04:32.971259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.971552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.971841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.972124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.972418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.972742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.973026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.973417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.973716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.974000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.974275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.974563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.974845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80538) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 02:04:32.995708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ab642a59c0 pid: 80537 (default app) Aug 08 02:04:32.999111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d422-059c-4c91-93be-2b5721b586c4 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:32.999396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d422-059c-4c91-93be-2b5721b586c4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:32.999667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62d422-059c-4c91-93be-2b5721b586c4 None None] PATH_INFO: `/v3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:33.003918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Mon Aug 8 02:04:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:33.008089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ab642a59c0 pid: 80538 (default app) Aug 08 02:04:41.643742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:41.648658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:41.648658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:41.648658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:41.653543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:41.653964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:41.654076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:41.725933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: INFO dbcounter [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] Registered counter for database keystone Aug 08 02:04:41.747306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG oslo_db.sqlalchemy.engines [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80538) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:04:41.753558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writer thread running {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:04:41.934630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG passlib.handlers.bcrypt [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80538) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:04:41.934957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG passlib.handlers.bcrypt [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80538) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:04:41.989258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:41.990861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:42.119469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1/3] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 08 02:04:42.153608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:42.226535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: INFO dbcounter [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] Registered counter for database keystone Aug 08 02:04:42.243361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG oslo_db.sqlalchemy.engines [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80537) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 02:04:42.250130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writer thread running {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 02:04:42.368197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11617f7c-e8a9-4215-a8e1-fdc6c392e21e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:42.373370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:42.373988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:42.374271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:42.374545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:42.382486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:42.436074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:04:42.436413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca91072-b166-4267-bdc1-7d930acbf722 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:42.445935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 3/4] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Aug 8 02:04:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:04:42.452573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:42.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21ede-8d7c-47d9-b7c4-67a260f9a0db None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:42.471365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:42.472466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:42.472779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:42.473055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] PATH_INFO: `/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:42.474485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] RBAC: Authorizing `identity:get_project(project_id=5f307f793c6c464eafa52c1ca17d9bc6)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:42.526231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 02:04:42.526578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8afd2b-0c19-400c-a506-e87156464863 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:42.531357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 2/5] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Aug 8 02:04:42 2022] GET /identity/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:44.498535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe403-83f7-4518-9a5b-596fb0e7597a None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:44.499085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe403-83f7-4518-9a5b-596fb0e7597a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:44.499085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe403-83f7-4518-9a5b-596fb0e7597a None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:44.503196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 4/6] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:44.508430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:44.508914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:44.509010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:44.531420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:44.532963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:44.541000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 3/7] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 02:04:44.563418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe403-83f7-4518-9a5b-596fb0e7597a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:44.577115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe403-83f7-4518-9a5b-596fb0e7597a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:44.581402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:44.582009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:44.582291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:44.582565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:44.591304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:44.593145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce130903-f0d3-42c4-b20e-a290966d5fba admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:44.602004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 5/8] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Aug 8 02:04:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:04:44.608280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:44.617370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087dd75-201e-4524-839a-5c8b8c4d2fc0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:44.621822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:44.622446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:44.622730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:44.623006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:44.624324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:44.626205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97e590-482c-46a8-88da-0bb3ac32b8da admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:44.627890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 4/9] 10.222.0.27 () {56 vars in 1198 bytes} [Mon Aug 8 02:04:44 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:46.680289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17528-a904-412e-be84-5ef32a516713 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:46.680839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17528-a904-412e-be84-5ef32a516713 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:46.680957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b17528-a904-412e-be84-5ef32a516713 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:46.682295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:46.687963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:46.688494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:46.688876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:46.699990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51cb26-83c9-46a4-8ba8-586e3d3ef5fa None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:46.700276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51cb26-83c9-46a4-8ba8-586e3d3ef5fa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:46.700554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51cb26-83c9-46a4-8ba8-586e3d3ef5fa None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:46.701887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 7/11] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:04:46.706973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:46.707260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:46.707542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:46.707694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:46.708919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:46.749830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG passlib.handlers.bcrypt [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80537) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 02:04:46.750259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG passlib.handlers.bcrypt [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80537) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 02:04:46.786917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 5/12] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:04:46.797226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:46.799824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:46.810695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:46.834762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:46.902415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 8/13] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:04:46.915232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57df32-ead2-4c05-8d71-d38773fdc655 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:46.923745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:46.949711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:46.950343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:46.950626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:46.950900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:46.961903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:04:46.970619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:04:46.971661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:04:46.972512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 6/14] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:04:46.977556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.033235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd317cd-78ab-4a43-b6da-1884d5a78f25 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.035498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.061046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.073254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.074220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.074527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.074842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.078468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8d38-471a-44d9-9fb3-3b9619fe49d0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.112047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.112654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.112967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.113246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.113922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:47.114226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.115623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.129255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 7/15] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:47.134778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.143462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.144693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.250054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 9/16] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:04:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:04:47.255929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.260158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.341694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0de21-2932-48dc-aa3e-043d70aed2f1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.359923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5829c82-4992-4766-a0b8-c4d0d0794936 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.375885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.376523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.376832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.377109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.378244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.379714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.381346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 8/17] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:04:47 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:47.387079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.393204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.394061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.394344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.394619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.402332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.413442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.414903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:04:47.433863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:04:47.434958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:04:47.435926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:04:47.435926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:04:47.435926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:04:47.437331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:04:47.443019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.487875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53632f-bfb7-413c-9cd0-5cbcf14887a1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.497644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.522176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.522738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.523021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.523304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.524429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.525916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.527604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:47.533350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:47.540949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3455f-6c21-40e8-a0fb-e74b6c3f54cf None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.577449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.578066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.578348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.578623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.579273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:47.579578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.581010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58cf67-9875-422a-a627-1201dfdef823 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.598818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:47.617740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:47.638924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfd32-cd81-4118-bd5e-be6f719947bc None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:47.677665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:47.678295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:47.678581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:47.678857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:47.679614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.domain.id=default, user.id=979e7eb5d5594f0ea4d2839dfb577133) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:47.679929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:47.681477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96346b-b011-40f2-aa2a-d842a3b2f784 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:47.689243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Aug 8 02:04:47 2022] GET /identity/v3/role_assignments?user.id=979e7eb5d5594f0ea4d2839dfb577133&scope.domain.id=default&role.id=c555140ef854486a808b99fb021d4982 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:49.961031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fd0-80f7-4465-b09e-5687d426a9ea None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:49.961520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fd0-80f7-4465-b09e-5687d426a9ea None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:49.961520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fd0-80f7-4465-b09e-5687d426a9ea None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:49.963000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:49.968102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b330-7018-4c20-ac8d-2489a94cbdd6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:49.968539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b330-7018-4c20-ac8d-2489a94cbdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:49.968739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35b330-7018-4c20-ac8d-2489a94cbdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:49.993657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-cc35b330-7018-4c20-ac8d-2489a94cbdd6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:49.994873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cc35b330-7018-4c20-ac8d-2489a94cbdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:50.082259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:50.115145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b0fd0-80f7-4465-b09e-5687d426a9ea None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:50.242013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244a7-d77e-4d71-a8c1-e3f8023e4cdd None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.242319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244a7-d77e-4d71-a8c1-e3f8023e4cdd None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.242672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028244a7-d77e-4d71-a8c1-e3f8023e4cdd None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.244029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:04:50.250173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.250459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.250738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.272598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b0fd0-80f7-4465-b09e-5687d426a9ea None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:50.287676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:50.288915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:50.317017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:50.349100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:50.349797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.350084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.350361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.361094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:50.362771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-99903c08-4ccc-47ad-ac47-aa9cfb6bdd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:04:50.378439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:04:50.379501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:04:50.380499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:04:50.380499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:04:50.380499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:04:50.381986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 13/25] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:04:50.383562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff241ca-29f6-49dc-ab3a-890a9b489597 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.384004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff241ca-29f6-49dc-ab3a-890a9b489597 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.384432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff241ca-29f6-49dc-ab3a-890a9b489597 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.385994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 14/26] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:50.394057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.394344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.394621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.414101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:50.415849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:50.470585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 13/27] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:04:50.471496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:50.481107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:50.506616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 15/28] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:50.509717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:50.683729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff980f3-f4cd-4c1a-9317-128a4f57b269 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:50.757362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:50.757901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.759379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.759661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.760324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:50.760636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:50.762219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:50.785345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:50.786591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:50.812591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:50.816963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799c464-ee1c-485e-90ab-1739bcbd10a0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:50.860331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:50.861254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.861689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.862097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.863388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:50.865456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:50.872933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:50.914884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea76ff-bb6f-4946-8be9-2c754a36337a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:50.950809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:50.951454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.951740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.952016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.962633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-916c76d4-3de3-41ec-9c1d-c24fa3f441e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:04:50.968064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:04:50.969784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 15/30] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:04:50.970769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993291-ebbd-4602-bbd5-2fb27d3b6dae None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:50.971459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993291-ebbd-4602-bbd5-2fb27d3b6dae None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:50.971459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba993291-ebbd-4602-bbd5-2fb27d3b6dae None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:50.972544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 16/31] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:50.976683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993291-ebbd-4602-bbd5-2fb27d3b6dae None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.088573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba993291-ebbd-4602-bbd5-2fb27d3b6dae None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.100005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 16/32] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Aug 8 02:04:50 2022] POST /identity/v3/roles => generated 219 bytes in 591 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:04:51.101796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.128268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:51.153573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.154237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.154526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.154916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.155600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.157312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901178b-5c02-48b6-a538-c71d7fb368b7 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.224479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643986-0aea-4f76-a13e-6e838da1d997 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.266327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.267331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.267883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.268367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.269403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:51.269909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.271651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273cf83-c747-4d90-baf8-786b940f3e83 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.287154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:51.288759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7f7f-49e2-4e2b-8460-ed13c6d65731 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.289045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7f7f-49e2-4e2b-8460-ed13c6d65731 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.289319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7f7f-49e2-4e2b-8460-ed13c6d65731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.310437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d32f7f7f-49e2-4e2b-8460-ed13c6d65731 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:51.312068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d32f7f7f-49e2-4e2b-8460-ed13c6d65731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:51.328252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:51.384119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 18/34] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:51.385342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b5eb-a839-410a-a886-3dacf23a2c70 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.385711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b5eb-a839-410a-a886-3dacf23a2c70 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.386159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b5eb-a839-410a-a886-3dacf23a2c70 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.387583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 19/35] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:51.388204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b5eb-a839-410a-a886-3dacf23a2c70 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.416014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 17/36] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:04:50 2022] POST /identity/v3/projects => generated 301 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:04:51.420310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.420883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.421384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.442933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:51.444890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:51.519238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b5eb-a839-410a-a886-3dacf23a2c70 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.532109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 18/37] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:51.532995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.570463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.571107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.571388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.571663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.573880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.576146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd1b4d-b0c6-4915-89a8-71c2b00f3434 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.577881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:04:51 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:51.579939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ed8-d8e2-493d-ac55-5e1299442b0c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.580426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ed8-d8e2-493d-ac55-5e1299442b0c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.580901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a2ed8-d8e2-493d-ac55-5e1299442b0c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.584809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 21/39] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:51.584809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2ed8-d8e2-493d-ac55-5e1299442b0c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.608651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:51.665642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fb79b-b9a9-4f60-93bb-2b7c5efb320a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.673775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:51.701019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.701642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.701928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.702228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.702883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.704483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.709239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a2ed8-d8e2-493d-ac55-5e1299442b0c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.742075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.742873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.743154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.743432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.744093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.745544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72379d44-ceb7-46d4-ac52-3a0ffa364e9c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.761695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 19/40] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Aug 8 02:04:51 2022] POST /identity/v3/roles => generated 225 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:04:51.762523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.819566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 22/41] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Aug 8 02:04:51 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:04:51.821182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.821647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.822063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.844702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:51.846535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:51.906972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5deb1f-2148-4037-a28c-6473f877673c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:51.942291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 23/42] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:51.955907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:51.956622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:51.956943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:51.957263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:51.958434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:51.959987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:51.962397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 20/43] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:51.970094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.970322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:51.982924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:52.016084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:52.090421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ebfa7-24ce-46d2-a7a1-7c1c88912c24 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:52.091576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9ffe4-6aec-471b-a20f-4fdee89372dc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:52.124427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:52.124596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:52.125129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:52.125259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:52.125417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:52.125539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:52.125700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] PATH_INFO: `/v3/domains/default/users/979e7eb5d5594f0ea4d2839dfb577133/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:52.125811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:52.126915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:52.127668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=979e7eb5d5594f0ea4d2839dfb577133, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:52.128357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:52.129245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e27b2-9e09-4214-b4bb-679334dc3cd1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:52.129936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 24/44] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:52.136191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:52.194941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 21/45] 10.222.0.27 () {58 vars in 1361 bytes} [Mon Aug 8 02:04:51 2022] PUT /identity/v3/domains/default/users/979e7eb5d5594f0ea4d2839dfb577133/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:04:52.213235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:52.237925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5856e-f534-49a8-8294-f51db71e9287 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:52.276088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:52.276764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:52.277046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:52.277317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:52.277980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:52.279487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca097f9c-e13c-4c2d-ad3e-719eb6cd8665 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:52.348732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 25/46] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:04:52 2022] POST /identity/v3/projects => generated 312 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:04:54.189349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b345b-da36-4d23-806b-c8b85c591234 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.189871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b345b-da36-4d23-806b-c8b85c591234 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.189953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b345b-da36-4d23-806b-c8b85c591234 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.191254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 22/47] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:54.196382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef785-bc08-4157-990d-96ee485c676a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.196841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef785-bc08-4157-990d-96ee485c676a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.197095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef785-bc08-4157-990d-96ee485c676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.215437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-650ef785-bc08-4157-990d-96ee485c676a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:54.216623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-650ef785-bc08-4157-990d-96ee485c676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:54.282602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 26/48] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:54.305424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b345b-da36-4d23-806b-c8b85c591234 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.344060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4d5f-b5f6-4fae-9531-d3706dadd79d None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.344349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4d5f-b5f6-4fae-9531-d3706dadd79d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.344628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a4d5f-b5f6-4fae-9531-d3706dadd79d None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.346011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 27/49] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:04:54.350792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.351077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.351354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.368949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:54.370670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:54.400393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:54.407884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b345b-da36-4d23-806b-c8b85c591234 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.424496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:54.438540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 28/50] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:04:54.441591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.442174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.442459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.442734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.453013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:04:54.458176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:04:54.460357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:04:54.461296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 23/51] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:04:54.461923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.465126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.560247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bdb74-b5dd-411c-9ae0-0f7a48bcc793 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.564414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b5481-19db-4471-a8e0-59aff3f32ffb None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.596021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.596684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.596976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.597254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.598378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:54.599820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:54.600104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.600739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.601024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.601314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.601418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 29/52] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:54.602276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:54.602583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:54.603951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:54.607702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.612890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 24/53] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:54.618221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.637354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:54.662115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:54.707037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6bfe7-6f91-43fe-8973-dc84f85a312e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.718254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b9b4-7f6e-4fc4-810b-4e3359eb5619 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.740773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.741378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.741654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.741933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.742572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:54.744045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:54.753040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.753669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.753970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.754227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.755363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:54.756838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:54.758416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 25/54] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:54.764287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:54.825021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 30/55] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Aug 8 02:04:54 2022] POST /identity/v3/projects => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:04:54.852523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:54.869159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecdf7-0436-4120-aff8-d8d33877b017 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:54.907832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:54.908514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:54.908857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:54.909140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:54.910322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:54.911806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cac8c-a5e6-4e5e-b5d7-b4ef740ed956 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:54.913709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 26/56] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:54.920801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:55.025893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8177-6f56-476f-aaaa-158ba43ced5c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:55.035334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:55.060458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:55.061164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:55.061450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:55.061730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:55.062506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.domain.id=default, user.id=979e7eb5d5594f0ea4d2839dfb577133) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:04:55.062822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:55.064285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924274c-4d6c-4072-9062-81c3cea6f151 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:55.074880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 31/57] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Aug 8 02:04:54 2022] GET /identity/v3/role_assignments?user.id=979e7eb5d5594f0ea4d2839dfb577133&scope.domain.id=default&role.id=c555140ef854486a808b99fb021d4982 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:04:56.842281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc371-a00e-4ed1-8852-2bee63cf4772 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:56.842774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc371-a00e-4ed1-8852-2bee63cf4772 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:56.842774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbc371-a00e-4ed1-8852-2bee63cf4772 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:56.844100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 27/58] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:04:56.849226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:56.849620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:56.849781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:56.868068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:04:56.869296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:04:56.933984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 32/59] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:04:56.958510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbc371-a00e-4ed1-8852-2bee63cf4772 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:57.060154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbc371-a00e-4ed1-8852-2bee63cf4772 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:57.098262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:57.098821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:57.099105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:57.099382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:57.100486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:57.101966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e827-ac3f-4382-a36a-80141012ffdb None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:57.103503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 28/60] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:04:57.109238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:04:57.205908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bdb25-9656-4332-a0d1-7f5cb6c8bcbc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:04:57.207361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:04:57.238408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:04:57.239003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:04:57.239284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:04:57.239633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:04:57.240475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:04:57.242243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af9784-d1b3-4ce3-ae6f-4b7e619a01ee None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:04:57.337323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 33/61] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:04:57 2022] POST /identity/v3/users => generated 275 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:00.412015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a2d9-c49c-491e-84d1-2c4305da9af5 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.412939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a2d9-c49c-491e-84d1-2c4305da9af5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.413364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403a2d9-c49c-491e-84d1-2c4305da9af5 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.414886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 29/62] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:00.423697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9090-7e94-434f-a700-80b4dbea4bcf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.424463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9090-7e94-434f-a700-80b4dbea4bcf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.424909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9090-7e94-434f-a700-80b4dbea4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.449740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d20e9090-7e94-434f-a700-80b4dbea4bcf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:00.450975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d20e9090-7e94-434f-a700-80b4dbea4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:00.543239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 34/63] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:00.584939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a2d9-c49c-491e-84d1-2c4305da9af5 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:00.623315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:00.626598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d9dd-cc14-4fc0-b8ee-e0edc630347c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.626891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d9dd-cc14-4fc0-b8ee-e0edc630347c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.627173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83d9dd-cc14-4fc0-b8ee-e0edc630347c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.628793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 35/64] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:00.649016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a9aa-9117-47b0-8e22-5a493dddfee1 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.649304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a9aa-9117-47b0-8e22-5a493dddfee1 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.649581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a9aa-9117-47b0-8e22-5a493dddfee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.685291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fac5a9aa-9117-47b0-8e22-5a493dddfee1 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:00.686435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fac5a9aa-9117-47b0-8e22-5a493dddfee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:00.744806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f403a2d9-c49c-491e-84d1-2c4305da9af5 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:00.806077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 36/65] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:00.814440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:00.815259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.815543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.815818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.836696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:00.840281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4ac3-1bc8-4b7d-b6f1-012fdff7acfd None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.840571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4ac3-1bc8-4b7d-b6f1-012fdff7acfd None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.840897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4ac3-1bc8-4b7d-b6f1-012fdff7acfd None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.842406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 37/66] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:00.844635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:00.845469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:00.846224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 30/67] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:00.852690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:00.852690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:00.852690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:00.856620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:00.872011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:00.873663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:00.888205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:00.945603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 38/68] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:00.946133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:00.946367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:00.980286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d49d74-5f86-45d0-b72b-9898976a04d1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.014770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.015613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.015897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.016167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.016936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:01.017163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.018524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47002-e917-424f-afac-d13e5d321336 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 31/69] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3808c-394c-477f-ae9d-3550652f2366 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3808c-394c-477f-ae9d-3550652f2366 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3808c-394c-477f-ae9d-3550652f2366 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 32/70] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:01.032229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3808c-394c-477f-ae9d-3550652f2366 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.063753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a1e43-bd37-48e7-8316-b8f82a151396 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.099805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.100442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.100748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.101027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.112378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:01.118594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:01.119485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:01.120784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 39/71] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:01.121906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.133009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.137781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3808c-394c-477f-ae9d-3550652f2366 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.170948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.171582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.171867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.172151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.182064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-0632e7db-e1df-413d-b343-687744c0820d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:01.187803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:01.188846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:01.189789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 33/72] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:01.190469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.190469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.190597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.197035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.208820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:01.210203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:01.224089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57264e8b-810b-4d30-aa15-c0c13c89f936 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.257675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.258301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.258582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.258858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.273789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.275228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7665c6-3197-45dd-b106-37b8b23e7d84 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.277358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 40/73] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.288746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 34/74] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:01.288898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078124d-c0c8-43f5-9785-37d9642e5f76 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.288898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078124d-c0c8-43f5-9785-37d9642e5f76 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.289075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.289159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078124d-c0c8-43f5-9785-37d9642e5f76 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.289159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 41/75] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:01.289159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078124d-c0c8-43f5-9785-37d9642e5f76 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.348387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.384681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe19d7a-dac1-4f7f-a4c0-02575ae9eb47 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.384925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078124d-c0c8-43f5-9785-37d9642e5f76 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.416719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.421585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.421993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.422193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.422468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.422603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.422745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.422882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.423215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.423410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:01.423722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.423906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:01.424265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.425144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.425587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f21dc-ce31-49d5-a15b-b55291c7437e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.433310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 35/76] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.437992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 42/77] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.438119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.438216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.438216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.438216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.454259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:01.455366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:01.520098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 43/78] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:01.521155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.545027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451309e9-d8d7-4686-8d6a-08bca38d7d0c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.578182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.579002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.579614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.579896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.580169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.623888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717a592-a352-4e22-8425-fd45f7dfe949 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.642383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.649524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.651674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.657843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.658514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.658797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.659068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.669745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:01.674869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:01.675703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:01.676450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 44/79] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:01.677115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.755259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 36/80] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:01.757583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.777491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aa10e-7755-43c2-8b93-ecd553119cce None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.794570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.810708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.811338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.811615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.811888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.813051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.814504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.816074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 45/81] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.816851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.849633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:01.856437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673b25e-524e-4c91-ab18-a7f4dbf2f018 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.890859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.891486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.891761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.892032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.893203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.894624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.896386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 37/82] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.896948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:01.920750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c86b-21ff-4ade-a18e-f9d64cd4ec46 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:01.955019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:01.955649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:01.955935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:01.956212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:01.957437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:01.958870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:01.960443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 46/83] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:01.961230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.000707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a131b3-2047-4c0b-b53e-8a6916751eff None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.016703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.036596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.037254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.037516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.037791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.038454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:02.038870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.040575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.052996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 38/84] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.057609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.061641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742950-6ee3-48e1-ba77-3c91cbc13654 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.082623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.094993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.097602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.097602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.097602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.097602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.098755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.100378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 47/85] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:01 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.101171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.162177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7aad-6264-4f49-87b7-edfe16ea8e1c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.196850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.197454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.197725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.198067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.200810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf5c0-abca-48fd-8f37-9a0c5f72d7ca None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.222503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.233955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.240793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.240793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.240793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.265375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.266917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.289944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.304169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.305709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.366766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 39/86] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:02.367777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.405870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 48/87] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:02.406712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.434175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.467841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a1dce-44c9-44e2-b5cc-b0f1cc12862c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.490768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.501999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.504224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.504224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.504224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.504224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.505231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.512488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c3be-7397-4cd0-8922-a5083ba2e90a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.546900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.547463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.547745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.548025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.549173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.550623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.552179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 49/88] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.552956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.594685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 40/89] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:02 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:02.595770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.664504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.677204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862f7d-66f7-4715-b640-174d7ab6584d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.710916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b103c53-5068-4b29-bcb0-d6e0d5400ba9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.715605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.716463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.716933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.717351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.718239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:02.718724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.720337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.726620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 50/90] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=43be248131c345f4b8664cc353124638 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.727721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.764714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.765686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.765969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.766243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.767480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.769090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.771128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 41/91] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.771870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.851520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:02.877588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ed084-61b4-43a3-95be-3904fb336c32 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.915802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab3098-9ce2-4a24-b937-7d21ce911255 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:02.924178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.925170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.925625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.926070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.927484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:02.929286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:02.932605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 51/92] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:02.933854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:02.961341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:02.962052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:02.962510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:02.962853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:02.966587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:03.057293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda49df-e51e-4059-b641-a65fda6e01f4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:03.059527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:03.066688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:03.118166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:03.118818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:03.119168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:03.119518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:03.121284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=979e7eb5d5594f0ea4d2839dfb577133) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:03.121665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:03.123232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:03.125495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:03.137062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 52/93] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/role_assignments?user.id=979e7eb5d5594f0ea4d2839dfb577133&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:03.137925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:03.185696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 42/94] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:03.196174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:03.287240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:03.295361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dcf89-8800-40b3-9fae-a39520cd0c61 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:03.334925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:03.335552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:03.335942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:03.340726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:03.340726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:03.340726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:03.340726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:03.354819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 53/95] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:03 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:03.417429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab7e44-1bff-4f88-8335-1707ecc31bb0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:03.469992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:03.471428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:03.471939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:03.472460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] PATH_INFO: `/v3/projects/447011ec7b8044aaa18018fedbd27826` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:03.477539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] RBAC: Authorizing `identity:get_project(project_id=447011ec7b8044aaa18018fedbd27826)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:03.479884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe44fb7-9eb8-4111-a2e4-e07d047b9362 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:03.481872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 43/96] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:03 2022] GET /identity/v3/projects/447011ec7b8044aaa18018fedbd27826 => generated 312 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:03.488292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:03.605714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce32092-55a6-49e8-966a-5559f3e8344b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:03.643328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:03.643954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:03.660830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:03.660830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:03.660830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=447011ec7b8044aaa18018fedbd27826, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:03.660830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:03.660830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eba2bb-f417-414d-b90b-b88cfefc4af6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:03.661353 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 54/97] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:03 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=447011ec7b8044aaa18018fedbd27826&role.id=43be248131c345f4b8664cc353124638 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:05.489089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdcd7a-573a-4632-916e-017c212ca0d0 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.490120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdcd7a-573a-4632-916e-017c212ca0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.490600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfdcd7a-573a-4632-916e-017c212ca0d0 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.492139 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 44/98] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:05.502537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.502998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.503136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.533661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:05.534855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:05.545673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:05.549776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70901d-a39d-4a15-b957-892cd346a8f2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.550291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70901d-a39d-4a15-b957-892cd346a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.550785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70901d-a39d-4a15-b957-892cd346a8f2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.552311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 45/99] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:05.557566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.558064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.558496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.589382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:05.590710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:05.607280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:05.660951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 55/100] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:05.703021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 46/101] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:05.714155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:05.735867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:05.861275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0094d1-9bfb-471e-9669-7d2af1cf38b2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:05.904682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5202b3-b7b7-4ddb-bec6-6bb1dc47b7a4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:05.909943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:05.910596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.910883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.911158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.930203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-d477ff78-e13e-44d0-9b19-34e99b76731d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:05.936843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:05.937825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:05.938840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:05.946927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 47/102] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:05.946927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bde3-7c90-42df-9b0f-a9bf9c0c7cf8 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.946927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bde3-7c90-42df-9b0f-a9bf9c0c7cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.947161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bde3-7c90-42df-9b0f-a9bf9c0c7cf8 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.949019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 48/103] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:05.958072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.958359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.958643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.974328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:05.974970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:05.975246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:05.975508 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:05.989048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:05.998927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:06.006640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:06.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:06.008203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:06.008865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:06.008865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:06.009497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 56/104] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:06.011481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.029021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.121214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 49/105] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:06.122311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.193378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9418b-3b9b-4d03-a29b-b95d44b8bb53 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.229558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.230462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.230889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.231305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.232115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:06.232566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.234121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a92f5-a282-48a1-bf95-de24eb32ce8b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.242461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd47ae-7dd2-4044-816a-12c064bb0743 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.242820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 57/106] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.244132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43b5c1-4e96-49e1-a1e9-98215fa027f6 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.244416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43b5c1-4e96-49e1-a1e9-98215fa027f6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.245620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43b5c1-4e96-49e1-a1e9-98215fa027f6 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.247457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 58/107] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:06.252433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43b5c1-4e96-49e1-a1e9-98215fa027f6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.280709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.281597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.282138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.282746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.283548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:06.283991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.285777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.288228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.296417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 50/108] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.297649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.322361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.359058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43b5c1-4e96-49e1-a1e9-98215fa027f6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.394854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.395438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.395769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.396053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.407860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:06.410741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e636426-5a8a-4bd3-b28b-9d2f8bb6abbe None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-37530c38-70bc-48d2-81da-552b01f6fd23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:06.413902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:06.414831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 59/109] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:06.415979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.416983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.416983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.435172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:06.436345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:06.456409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.457320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.457861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.458272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.459814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.462136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.464581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 51/110] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.465632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.511705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 60/111] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:06.516987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.547998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.587706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583bc8c-c948-4a75-95a5-012e24f77528 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.595010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.626898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.627541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.628098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.628382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] PATH_INFO: `/v3/users/7e95a863bbe042f2a5f10db4fba3c751` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.641643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91183e64-fc75-4ec8-a35f-8c022c2d7533 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.643323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] RBAC: Authorizing `identity:get_user(user_id=7e95a863bbe042f2a5f10db4fba3c751)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.644960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.646729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 52/112] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/users/7e95a863bbe042f2a5f10db4fba3c751 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.647304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.679735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.680341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.680622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.680921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.681584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:06.681887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.683283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.692952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 61/113] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.694157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.756152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bd5c8-f36b-4c7c-8751-54d4e2a5883a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.779917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.790892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.796304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.796304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.796304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.796304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.796304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a30a2-5dd6-48e7-9975-5ced08fe1603 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.801910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 53/114] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.801993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485781-f9c1-4c02-ac40-73a0cf17742f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.802077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854313a0-979f-492b-a41f-1b10757ad9de None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.802077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854313a0-979f-492b-a41f-1b10757ad9de None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.802077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854313a0-979f-492b-a41f-1b10757ad9de None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.802077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 54/115] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:06.804042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854313a0-979f-492b-a41f-1b10757ad9de None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.814734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:06.831831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.832462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.832772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.833039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.834179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.835610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.837234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 62/116] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.837983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:06.911985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854313a0-979f-492b-a41f-1b10757ad9de None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.941365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07915f-aef7-43ed-b18e-d3053a641107 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:06.944917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.945523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.945803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.946075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.974312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:06.974930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.975211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.975487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.976603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:06.978059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3448764-8d40-4bbf-8ff2-4d8727721c06 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:06.980128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 63/117] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:06.981104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57637b1-6fd5-4d73-a5ee-f856aa07590f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:06.981381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57637b1-6fd5-4d73-a5ee-f856aa07590f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:06.981652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57637b1-6fd5-4d73-a5ee-f856aa07590f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:06.982954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 64/118] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:06.983498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57637b1-6fd5-4d73-a5ee-f856aa07590f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.004940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.014336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.015839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528abee-8ed5-4b54-9749-4fb2c2cd03a2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.041557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.086070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57637b1-6fd5-4d73-a5ee-f856aa07590f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.115621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 55/119] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:07.117034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.117319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.117593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.119611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.120373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.120544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.120853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.122809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, user_id=a1af2ddf4f644c50b16fc01349ed03ce, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.124229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.134936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:07.136708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:07.208507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 65/120] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:06 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:07.208507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.208799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 56/121] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:07.208799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.243315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.319402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f7d5b-0605-4c0a-a4ec-e24a56b0b4ad None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.329485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4f96d-562e-461f-93ee-5f24d8c1aa1d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.336073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.364318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.364985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.365278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.365560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.366217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.367700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.375519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.376201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.376538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.376843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.378249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.379469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305fb48-b7cc-4136-81e9-46bb249bc94b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.381358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 57/122] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:07.382323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.382599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.382920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.402764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:07.404025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:07.442444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 66/123] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:05:07 2022] POST /identity/v3/projects => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:07.443156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.497717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 58/124] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:07.498456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.527896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.591909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19934b8d-abf3-45d8-ba2e-53a0c1a75447 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.640803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.641308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.645437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.645437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.645437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.645437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.646201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 67/125] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:07.647033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.648959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab006e1b-b34c-474a-b62c-235e045c152c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.686063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.689907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.690643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.690923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.691246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.702714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:07.707671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:07.708544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:07.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 59/126] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:07.710399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.795686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48482607-7177-4a61-adac-6b696f28c90b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.802185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:07.824524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69a320-3849-41a5-8450-2c430ac26039 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:07.839305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.839988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.840267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.840547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/979e7eb5d5594f0ea4d2839dfb577133/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.843067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, user_id=979e7eb5d5594f0ea4d2839dfb577133, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:07.844849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:07.864095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:07.868442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:07.868442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:07.868442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:07.876705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:07.881485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:07.882382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:07.883128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 60/127] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:07.883744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:07.940998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 68/128] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:07 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/979e7eb5d5594f0ea4d2839dfb577133/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:07.941968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.002838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f382450-4af1-4b52-bf4d-a86d09b40143 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.068924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:08.074676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d573b7f-256c-4f83-bb80-7cf44ec3a0b5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.079616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.080304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.083839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.083839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.083839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=7e95a863bbe042f2a5f10db4fba3c751) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:08.083839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.083839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.098269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 61/129] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/role_assignments?user.id=7e95a863bbe042f2a5f10db4fba3c751&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=aaf25a175bb641748e1620292b72a5cb => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:08.099145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.106924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:08.116434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.117151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.118310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.118310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.118687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:08.118687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.121479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.133373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 69/130] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:08.140092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.273533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c4ee7-e2ad-4e4d-b885-6180821865f2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.316877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.317524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.317803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.318077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.319322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.320752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.322515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 70/131] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:08 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:08.328170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.355364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee416c-a37d-48c7-bab4-eb2e5f567897 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.363474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:08.440842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d9f31-ffad-475e-b47c-b38fb2275025 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.445482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a942617-a201-4247-a461-061300929704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.446129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942617-a201-4247-a461-061300929704 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.446419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942617-a201-4247-a461-061300929704 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.446697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a942617-a201-4247-a461-061300929704 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.447389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a942617-a201-4247-a461-061300929704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:08.447813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a942617-a201-4247-a461-061300929704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.449367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a942617-a201-4247-a461-061300929704 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.470958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 62/132] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:08.481711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.481912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a942617-a201-4247-a461-061300929704 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.482451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.482741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.483020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] PATH_INFO: `/v3/projects/447011ec7b8044aaa18018fedbd27826` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.484168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] RBAC: Authorizing `identity:get_project(project_id=447011ec7b8044aaa18018fedbd27826)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.485770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.487666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 71/133] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:08 2022] GET /identity/v3/projects/447011ec7b8044aaa18018fedbd27826 => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:08.493617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.592493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:08.651676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:08.655475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a942617-a201-4247-a461-061300929704 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.674699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a21-b13b-43bd-82bd-09c19d9e1a6d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.719743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.720384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.720680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.720960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.725679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.727304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.729475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 63/134] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:08 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:08.736274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:08.761415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.762114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.762411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.762690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] PATH_INFO: `/v3/projects/447011ec7b8044aaa18018fedbd27826/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.764768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] RBAC: Authorizing `identity:create_grant(project_id=447011ec7b8044aaa18018fedbd27826, user_id=a1af2ddf4f644c50b16fc01349ed03ce, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.766259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.856993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 72/135] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:08 2022] PUT /identity/v3/projects/447011ec7b8044aaa18018fedbd27826/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:08.880552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea39c32-2efd-414f-9dd5-689a76d9985c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:08.937683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:08.938368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:08.938651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:08.938928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:08.940293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:08.941785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72992e6e-ebd6-4ead-8d38-0b9583693e74 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:08.943448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 64/136] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:08 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:08.950032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:09.106683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bccd-20a6-4f32-93cf-852b88977a98 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:09.155217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:09.155819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:09.156105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:09.156389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:09.162488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, user_id=a1af2ddf4f644c50b16fc01349ed03ce, role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:09.163991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a9e96-73bf-4bbf-b83c-b7f1c7889b78 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:09.240871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 73/137] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:08 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/a1af2ddf4f644c50b16fc01349ed03ce/roles/47edb61814514a2b8e3e7b38f680c3da => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:09.439882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261d03-8809-41bc-96f1-509d84b0fe44 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:09.440051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261d03-8809-41bc-96f1-509d84b0fe44 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:09.440339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4261d03-8809-41bc-96f1-509d84b0fe44 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:09.441867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 65/138] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:09.450650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:09.450963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:09.451215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:09.481801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:09.483052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:09.491659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:09.568358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 74/139] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:09.609773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4261d03-8809-41bc-96f1-509d84b0fe44 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:09.612618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:09.729290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4261d03-8809-41bc-96f1-509d84b0fe44 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:09.774019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:09.774654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:09.774937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:09.775210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:09.787599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:09.793889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:09.794758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:09.795547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 66/140] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:09.801129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:09.927047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b972919-a368-401e-a3d0-a5e721cb22d1 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:09.962440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:09.963599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:09.963887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:09.964220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:09.964924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:09.965252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:09.966644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:09.975467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 75/141] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:09.981194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:10.053529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:10.103754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432a751-0a71-49fa-b200-8f49f213451a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:10.154217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:10.155237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.155734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.156225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.157655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:10.159458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:10.161285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 67/142] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:09 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:10.167603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:10.173717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:10.301775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c183e-6938-4e32-a882-986f93dd7865 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:10.342858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:10.343472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.343796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.344072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.428699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:10.430232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:10.433882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:10.554300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 76/143] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:10.560087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:10.713902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5db271-fbea-441a-9089-bde0c165e4c2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:10.750147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:10.755839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:10.756515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.757174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.757450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.769046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:10.769046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d393b-3392-48e3-be28-5baa92fe9e55 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:10.769046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 68/144] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:10 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:10.769303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:10.781693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d993686-8aab-4b07-bcad-57b7c112dd51 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.781971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d993686-8aab-4b07-bcad-57b7c112dd51 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.782253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d993686-8aab-4b07-bcad-57b7c112dd51 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.785254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 69/145] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:10.793100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad79b5-967b-4deb-a30e-41a6231cd43a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.793388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad79b5-967b-4deb-a30e-41a6231cd43a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.793665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad79b5-967b-4deb-a30e-41a6231cd43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.833156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-35ad79b5-967b-4deb-a30e-41a6231cd43a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:10.834277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-35ad79b5-967b-4deb-a30e-41a6231cd43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:10.945454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b70c33-9197-4833-9168-9b5163cb0ab0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:10.950588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:10.956613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 70/146] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:10.958674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562409-8e14-4955-84bf-78f4b1c783d7 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.963287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562409-8e14-4955-84bf-78f4b1c783d7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.963287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562409-8e14-4955-84bf-78f4b1c783d7 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:10.963287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 71/147] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:10.978367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:10.979484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:10.979971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.011528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:11.012761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:11.016130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.016778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.017114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.017392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.018172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:11.018487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:11.020014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:11.029085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 77/148] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:10 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=43be248131c345f4b8664cc353124638 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:11.036516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.170310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 72/149] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:11.227508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.237895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75247d0f-aec6-4368-bcae-5961357a98a9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:11.308026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.309247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.309802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.310312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.312877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:11.314639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:11.316522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 78/150] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:11.322164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.332517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:11.389105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e274755-3106-407c-aa07-d0b5f32311b5 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:11.393895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:11.436888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.437504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.437791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.438063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.449399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-38dfed64-2815-41ba-9577-b4d653c6b0a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:11.454390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:11.455247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:11.456437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 73/151] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:11.457137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e490d-80e4-436e-b5da-c9b1bee7080e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.457386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e490d-80e4-436e-b5da-c9b1bee7080e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.457550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e490d-80e4-436e-b5da-c9b1bee7080e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.458894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 74/152] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:11.463881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e490d-80e4-436e-b5da-c9b1bee7080e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.487415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22d32-1bdb-4912-b34c-a53c310347e9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:11.532768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.533400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.533676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.533949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.534904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:11.536361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a82ac-28fa-4039-9360-ba3602477953 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:11.605635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e490d-80e4-436e-b5da-c9b1bee7080e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:11.624512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:11.643141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.643895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.644331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.644812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.645620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:11.645928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:11.647448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:11.650059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 79/153] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:11 2022] POST /identity/v3/users => generated 283 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:05:11.654676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.655835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.656809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.658850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 75/154] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:11.663537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.685135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:11.687843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:11.778053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 80/155] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:11.806775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cae6e5-ad22-4884-9fba-6b8a40455770 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:11.826133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.855599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:11.857201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:11.857491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:11.857763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] PATH_INFO: `/v3/users/7e95a863bbe042f2a5f10db4fba3c751` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:11.862314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] RBAC: Authorizing `identity:get_user(user_id=7e95a863bbe042f2a5f10db4fba3c751)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:11.865219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:11.869242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 76/156] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/users/7e95a863bbe042f2a5f10db4fba3c751 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:11.874395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:11.881629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:11.949418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:11.977110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fca0b6-eef3-4b12-ac76-7ed8480f5495 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.013238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c469be3-dbfc-4fdc-83be-5e674644d97a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.018817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.019459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.019741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.020015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.033635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:12.038740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:12.040229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:12.041045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 81/157] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:12.047030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.072414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.073140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.073521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.073793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.075380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.077161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.079559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 77/158] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:11 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:12.089692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.149390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.170707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b99624-3d9d-419e-9d46-bdbbd082fb7a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.211341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.211889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.212170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.212445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.213790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:12.214108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b7622-cd80-42ac-923a-14945cc4a117 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.217669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e220b68-ea27-468d-ab41-cebd468faecf None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.225620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 82/159] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:12.228073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19a8-02aa-45af-af83-43c3b17b261e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.228607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19a8-02aa-45af-af83-43c3b17b261e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.229243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e19a8-02aa-45af-af83-43c3b17b261e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.230768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 83/160] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:12.231684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e19a8-02aa-45af-af83-43c3b17b261e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.241451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.267936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.268619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.269818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.270101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/7e95a863bbe042f2a5f10db4fba3c751/roles/aaf25a175bb641748e1620292b72a5cb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.272162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, user_id=7e95a863bbe042f2a5f10db4fba3c751, role_id=aaf25a175bb641748e1620292b72a5cb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.273755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc948c-0fd0-48d8-a2d1-41d7e15d919e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.350438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 78/161] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:12 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/users/7e95a863bbe042f2a5f10db4fba3c751/roles/aaf25a175bb641748e1620292b72a5cb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:12.351671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b368f-ed7b-4000-8d1a-86b776084df7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.351956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b368f-ed7b-4000-8d1a-86b776084df7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.352240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b368f-ed7b-4000-8d1a-86b776084df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.354734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e19a8-02aa-45af-af83-43c3b17b261e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.374054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7f4b368f-ed7b-4000-8d1a-86b776084df7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:12.375345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7f4b368f-ed7b-4000-8d1a-86b776084df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:12.397561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.398572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.398988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.399454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] PATH_INFO: `/v3/users/979e7eb5d5594f0ea4d2839dfb577133` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.401406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=979e7eb5d5594f0ea4d2839dfb577133)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.403233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd168c0-bb6c-4a03-8390-0bf84780f9c1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.405094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 84/162] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/users/979e7eb5d5594f0ea4d2839dfb577133 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:12.406504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4cdb6-58ff-4c4f-8a00-2e7bd0d16a25 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.407000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4cdb6-58ff-4c4f-8a00-2e7bd0d16a25 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.407471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4cdb6-58ff-4c4f-8a00-2e7bd0d16a25 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.408840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 85/163] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:12.410921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4cdb6-58ff-4c4f-8a00-2e7bd0d16a25 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.431642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.460336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 79/164] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:12.461550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.461789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.462191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.494368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:12.495621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:12.546979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4cdb6-58ff-4c4f-8a00-2e7bd0d16a25 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.583471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 80/165] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:12.584195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.619047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.620659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.621152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.621599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.629256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.709674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6e89c-bfed-48f4-945f-ea9237a69a1b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.718282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.720343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.740147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.757792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.758443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.758720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.758989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.780648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:12.793719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:12.794630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:12.795605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 81/166] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:12.796252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.835715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 86/167] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:12.836634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:12.940434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f8eea-aa6d-4458-856f-fafd589a8302 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.951004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.958626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebd2a5-e0eb-4e9b-b3e3-a2a69b8898fd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:12.965863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:12.980516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:12.981190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:12.981574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:12.981943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:12.983191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:12.983593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:12.985100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:12.993778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 87/168] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:12.996078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.057535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.058291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.058571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.058846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.101563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4d6ff-3bd5-4546-a55c-7ae26f3f3e48 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.105330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:13.110229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:13.111183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:13.111945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 82/169] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:13.112622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.151737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.152434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.152742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.153020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.154194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.155787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:13.157763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 88/170] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:12 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:13.158586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.239020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:13.329465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3120574-fde9-4509-a493-98cb79a15b27 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.390906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbc0df-c3c9-4804-bda2-b402a46c3db2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.405394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.406057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.406342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.406621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.407861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.412693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:13.412693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 83/171] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:13.412693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.452759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:13.457189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.458055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.458555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.459027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.460077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:13.460583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.462277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:13.479165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 89/172] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:13.480347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.553194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ed5d-2ae2-4fca-afc9-e30b6f5995b1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.605401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.606038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.606324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.606599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.607344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=979e7eb5d5594f0ea4d2839dfb577133) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:13.607659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.609137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:13.615543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 84/173] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/role_assignments?user.id=979e7eb5d5594f0ea4d2839dfb577133&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:13.621986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.736868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e31b7-ebd9-4223-8564-ecf99abb5e65 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.745971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:13.808240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:13.822697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.825410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.825410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.825410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.829965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7fa7f-5f7a-4156-aa42-baa80a706909 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:13.902123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:13.902831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:13.903106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:13.903373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] PATH_INFO: `/v3/users/a1af2ddf4f644c50b16fc01349ed03ce` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:13.907500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] RBAC: Authorizing `identity:get_user(user_id=a1af2ddf4f644c50b16fc01349ed03ce)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.908989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:13.910678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 85/174] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/users/a1af2ddf4f644c50b16fc01349ed03ce => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:13.917074 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:13.959758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:13.961807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:14.126944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ac6da-efeb-47e0-a437-f3bd08162c58 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:14.143103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 90/175] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:14.154992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:14.169232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:14.202278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:14.202942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:14.203222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:14.203486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:14.257215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:14.342008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:14.343501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:14.387084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0527ed-cfae-4b88-82b7-3735dadfef3a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:14.464423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:14.465110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:14.465393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:14.465665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] PATH_INFO: `/v3/projects/447011ec7b8044aaa18018fedbd27826` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:14.469474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=447011ec7b8044aaa18018fedbd27826)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:14.470967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:14.472709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 91/176] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:14 2022] GET /identity/v3/projects/447011ec7b8044aaa18018fedbd27826 => generated 312 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:14.479698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:14.530455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 86/177] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:05:14.535991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:14.631910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:14.667074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:14.691113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c410035-5045-48c7-b173-bdd9958c225b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:14.743437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e63c77-aa41-479b-b3c5-f9ce9debb4b4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:14.747538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:14.748237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:14.748522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:14.748819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:14.750035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:14.751997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:14.753785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 87/178] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:14 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:14.760608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:14.825315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:14.825947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:14.826234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:14.826511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:14.827254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=447011ec7b8044aaa18018fedbd27826, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:14.827568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:14.829062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda991de-638a-4ddf-9456-f1b11baf419e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:14.847040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 92/179] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:14 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=447011ec7b8044aaa18018fedbd27826&role.id=43be248131c345f4b8664cc353124638 => generated 608 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:14.930260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913318b-9319-403b-bce4-edc78fef7293 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:14.968645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:14.987533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:14.987533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:14.987533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:14.987533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:14.987942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=a1af2ddf4f644c50b16fc01349ed03ce) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:14.988021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:14.990922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad834d-a692-4bd5-b2cb-de10d3442612 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:15.002108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 88/180] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:14 2022] GET /identity/v3/role_assignments?user.id=a1af2ddf4f644c50b16fc01349ed03ce&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:15.146725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9937f4-12e6-4592-8b36-06f1a3d0ef72 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.147234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9937f4-12e6-4592-8b36-06f1a3d0ef72 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.147307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9937f4-12e6-4592-8b36-06f1a3d0ef72 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.148897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 93/181] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:15.155110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782f8c5-bede-4138-831d-3ad8f5ec9939 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.155624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782f8c5-bede-4138-831d-3ad8f5ec9939 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.155700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782f8c5-bede-4138-831d-3ad8f5ec9939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.185992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3782f8c5-bede-4138-831d-3ad8f5ec9939 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:15.187168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3782f8c5-bede-4138-831d-3ad8f5ec9939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:15.266462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 89/182] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:15.291089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9937f4-12e6-4592-8b36-06f1a3d0ef72 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.344878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4eedf-6658-4ffb-b29e-c245979196fc None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.345282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4eedf-6658-4ffb-b29e-c245979196fc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.345814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4eedf-6658-4ffb-b29e-c245979196fc None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.347195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 90/183] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:15.360389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.360692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.360976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.385415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:15.386828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:15.423314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9937f4-12e6-4592-8b36-06f1a3d0ef72 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:15.460771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:15.461336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.461745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.462036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.469107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:15.471923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 91/184] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:15.476047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:15.480962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:15.481836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:15.483014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 94/185] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:15.488595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.497643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.566674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:15.644422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4ae0a-af89-4b1e-831a-c24bc6469c5a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:15.644869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc25e1f-69cb-42c2-ac9a-4506dbce2299 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:15.681019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:15.682312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.682651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.682930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.683616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:15.683926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:15.685380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ea1be-aa88-4786-91d4-513241ca1ef6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:15.686427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:15.687274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.687762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.688227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.694126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 92/186] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:15.695164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7c443-7e75-4d6a-acae-ec4a0c5a86e7 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.695451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7c443-7e75-4d6a-acae-ec4a0c5a86e7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.695844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7c443-7e75-4d6a-acae-ec4a0c5a86e7 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.697262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 93/187] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:15.699605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7c443-7e75-4d6a-acae-ec4a0c5a86e7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.706700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a0a783af-41b0-4dca-b673-753684ae29ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:15.712139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:15.713116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:15.714808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 95/188] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:15.719389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.719680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.719955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.738414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:15.739668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:15.795942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:15.811171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7c443-7e75-4d6a-acae-ec4a0c5a86e7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:15.812568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 96/189] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:15.816732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.822060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:15.847113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:15.847740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.848013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.848283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.861888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:15.863236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d8f38-b1a5-4aaa-84b5-e93fc2236d0c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:15.864865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 94/190] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:15.865992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853a16-2d8c-4719-9305-42a2eefdb23a None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.866268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853a16-2d8c-4719-9305-42a2eefdb23a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.866718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09853a16-2d8c-4719-9305-42a2eefdb23a None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.872477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 95/191] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:15.872477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09853a16-2d8c-4719-9305-42a2eefdb23a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.919699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52480535-1e55-4594-a35f-759f0cf0035d None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:15.952933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:15.953550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:15.953825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:15.954101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:15.954767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:15.955073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:15.956448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:15.965090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 97/192] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:15.965813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:15.974875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09853a16-2d8c-4719-9305-42a2eefdb23a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.008618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.009283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.009693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.009993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.012813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.020317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-51b3f08a-2141-4e7a-b43f-9a5f130f987e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:16.025846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:16.028135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:16.028946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 96/193] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:16.029872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.030158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.030431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.048070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:16.049317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:16.064871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578c60b-2d29-46b4-859b-84dec6dbc385 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.072637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.099800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.100399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.100689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.100966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.116136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 97/194] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:16.128815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.182438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.183916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.216097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd9e08-e113-4752-8d9b-32aa94c3845a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.231610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.249637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.250262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] PATH_INFO: `/v3/users/7e95a863bbe042f2a5f10db4fba3c751` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] RBAC: Authorizing `identity:get_user(user_id=7e95a863bbe042f2a5f10db4fba3c751)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 98/195] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/users/7e95a863bbe042f2a5f10db4fba3c751 => generated 289 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.256797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.282691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 98/196] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:16.283442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.288324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.361867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb18ef-0b0a-43f0-a426-336b13969d86 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.381720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83621b74-e74d-42d7-8932-c0def4de27a1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.396235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.396876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.397156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.397429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.398098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:16.398399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.399767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.408233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 99/197] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.408840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.417565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.418600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.418878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.419151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.420265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.421734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.427790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 99/198] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.427790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.441643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.503810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.515955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27480560-bc5f-4e49-a670-dc80eece676c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.522810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2b6-8f64-4151-8bd7-7e02830ed40d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.550780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.551381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.551658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.551927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.556366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.557312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.557597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.557872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.558996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.560434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.562266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 100/199] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.562833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.624094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.625575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.651339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.666580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac86bd-0768-4d68-9826-38e1a6aeaf3c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.701555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.702175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.702450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.702716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.703843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.705331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.706887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 101/200] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.711768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.718126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.730921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 100/201] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:16.731726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.807090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164f14c-3ac8-4bad-ac8f-9e0a25e11920 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.831598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70b-f5c1-4c15-854d-3cee2a30eaca None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:16.840865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.841469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.841751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.842017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.842653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.844082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.865690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:16.866288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:16.866565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:16.866835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:16.867556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=f3d95545a2b5474a84964cb441014cc1) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:16.867861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:16.870360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:16.875497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 101/202] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/role_assignments?user.id=f3d95545a2b5474a84964cb441014cc1&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:16.876337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.935402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:16.943747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 102/203] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:16 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:16.944953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:16.960113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:17.016094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10312b1-8170-4370-ab95-a220b3a3c536 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:17.071050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:17.071769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:17.072175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:17.072539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:17.101574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363957d-f007-4024-985f-eb0b1020daa6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:17.159905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:17.160559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:17.161265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:17.161695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:17.163086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:17.164773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:17.167816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 103/204] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:17.173762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:17.174138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:17.175320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:17.231191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:17.265373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:17.293570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 102/205] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:17.297936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b091a-d0b4-4695-9872-d2245775c6c4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:17.300295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:17.341681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:17.342243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:17.342510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:17.342934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:17.343991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26, user.id=7e95a863bbe042f2a5f10db4fba3c751) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:17.344436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:17.346047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:17.354346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 104/206] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:17 2022] GET /identity/v3/role_assignments?user.id=7e95a863bbe042f2a5f10db4fba3c751&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=aaf25a175bb641748e1620292b72a5cb => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:17.428058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7039d7a-ee7b-4305-b206-084a854f2113 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:17.467614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:17.468328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:17.468612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:17.468913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:17.470076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:17.471533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea10f8c-8152-4f4c-822e-893446283001 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:17.473554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 103/207] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:17 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:17.482109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:17.622717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a032ad-7c4c-4f56-af10-a43c1a6af638 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:17.667333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:17.668354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:17.668876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:17.669342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:17.670313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=f3d95545a2b5474a84964cb441014cc1) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:17.670828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:17.674375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463792d-a29b-4b67-afaf-6b3ff8e46a6e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:17.685208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 105/208] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:17 2022] GET /identity/v3/role_assignments?user.id=f3d95545a2b5474a84964cb441014cc1&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:19.033528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7b66f-94d5-4ea5-8073-1a0334ed3929 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.034126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7b66f-94d5-4ea5-8073-1a0334ed3929 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.034126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7b66f-94d5-4ea5-8073-1a0334ed3929 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.035652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 104/209] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:19.042177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa50b-d016-4392-ade2-e4b149808218 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.042618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa50b-d016-4392-ade2-e4b149808218 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.042618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa50b-d016-4392-ade2-e4b149808218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.050882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.061548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2afaa50b-d016-4392-ade2-e4b149808218 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:19.062735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2afaa50b-d016-4392-ade2-e4b149808218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:19.132052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 106/210] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:19.155404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7b66f-94d5-4ea5-8073-1a0334ed3929 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.179336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.265093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7b66f-94d5-4ea5-8073-1a0334ed3929 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.301325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.301887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.302172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.302506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.314146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:19.318988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:19.319864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:19.320655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 105/211] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:19.322266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfaf63-a9dd-4f0d-8154-97af70296195 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.323560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfaf63-a9dd-4f0d-8154-97af70296195 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.324039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfaf63-a9dd-4f0d-8154-97af70296195 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.325909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 107/212] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:19.329767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.331178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a8b45-2336-41d0-b4ba-245711ccda20 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.331659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a8b45-2336-41d0-b4ba-245711ccda20 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.332143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a8b45-2336-41d0-b4ba-245711ccda20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.351095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e30a8b45-2336-41d0-b4ba-245711ccda20 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:19.352625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e30a8b45-2336-41d0-b4ba-245711ccda20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:19.413572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.432714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 108/213] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:19.432714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f27e-d87d-4384-bec6-f69754e45ff2 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.432714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f27e-d87d-4384-bec6-f69754e45ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.432714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066f27e-d87d-4384-bec6-f69754e45ff2 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.432714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 109/214] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:19.437463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.437906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.438268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.440134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb54f19-533c-4013-8d73-f871e918f622 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.459339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:19.460961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:19.477126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.477737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.478014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.478812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.479481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:19.479783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:19.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:19.489552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 106/215] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:19.489552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.532437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.540493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 110/216] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:19.542132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.618576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e74f1-b92e-4197-9a38-7f7f5eac4bac None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.649634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cc1f4-f0d9-4f1a-bc76-7f92b3a53bc2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.657332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.657969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.658248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.658526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.671284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:19.678073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:19.680438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:19.681837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 107/217] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:19.682494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.687515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.693630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.694268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.694545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.694818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.695940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:19.697696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:19.699285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 111/218] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:19.700032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.767368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.797151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27cf68-801d-4ffa-99e3-f20a1840900f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.806881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357251f-343a-4b13-94a8-885f8a5752f7 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.830070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.830693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.830975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.831245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.832347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:19.833841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:19.835396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 108/219] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:19.836276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.842148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.842761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.843034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.843300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.843956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:19.844253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:19.845649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3f68c-8b1d-4f30-a16e-5ce4fb00fac6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:19.856730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 112/220] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:19.857977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910c07-836e-463b-ac4f-8eabd277b66f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.858257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910c07-836e-463b-ac4f-8eabd277b66f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.858526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f910c07-836e-463b-ac4f-8eabd277b66f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.859922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 113/221] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:19.861429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f910c07-836e-463b-ac4f-8eabd277b66f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.906301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.936367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e424d2-bf2b-4296-8abc-69f1fedcc628 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.962688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f910c07-836e-463b-ac4f-8eabd277b66f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:19.969572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.970294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.970577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.970851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.972216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:19.973700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:19.975319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 109/222] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:19.976093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:19.986833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:19.997422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:19.998037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:19.998317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:19.998590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:19.999234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.000684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36630e-954b-4417-94d2-b667808504f2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.079762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099825bc-e857-4ba8-b86c-b3567cd1e19d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.092194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 114/223] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:19 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:20.110240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.110604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.110891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.120728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.121388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.121666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.121933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] PATH_INFO: `/v3/users/24c2ba67fef340948c7eec27a92d0a28` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.131400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:20.140685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] RBAC: Authorizing `identity:get_user(user_id=24c2ba67fef340948c7eec27a92d0a28)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.142248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.142701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:20.143940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:20.146366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 110/224] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:19 2022] GET /identity/v3/users/24c2ba67fef340948c7eec27a92d0a28 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:20.149629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.232723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 115/225] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:20.232723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.275128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdee88-0918-4be4-a95f-3c58d4d46223 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.312846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.313514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.313793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.314066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/f3d95545a2b5474a84964cb441014cc1/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.315998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, user_id=f3d95545a2b5474a84964cb441014cc1, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.317495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.355678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befdc67-3cc3-4a2f-af8f-8806d38d7b15 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.366832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:20.394755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.395362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.395500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 111/226] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:20 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/f3d95545a2b5474a84964cb441014cc1/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:20.395723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.396000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.396200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.488034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:20.499418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.502435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.542647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2608-4dea-482e-b132-26d76c2f3fc3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.589395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.590169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.590979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.591358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.606266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:20.612289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:20.613390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:20.614375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:20.616553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 112/227] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:20.624115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.653399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 116/228] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:20.660527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.679880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:20.766400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e87e8d-9bff-460f-9040-efa8bd7593f1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.785011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:20.795528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87fc0-b3f8-47eb-b797-ebb3947cef5d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:20.834845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.836316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.837292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.838115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.839532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:20.840338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:20.841216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:20.841742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.843341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.843894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:20.844372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:20.845757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:20.849765 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:20.851644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 117/229] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:20.858982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.870744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 113/230] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:20.872157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:20.999717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10aa2c-2544-4d10-8d37-c49a722d4ff0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:21.020151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:21.042827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:21.044389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:21.044920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:21.045503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:21.046910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:21.048411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:21.050626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 114/231] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:21.056437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:21.075438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19643b-dc94-4548-ad0b-a69a59caa337 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:21.106247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:21.151446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:21.151964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:21.152243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:21.152512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:21.153270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=24c2ba67fef340948c7eec27a92d0a28) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:21.153660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:21.155091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:21.163330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c530165-7e51-446a-bc3d-ebcd26549903 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:21.170616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 118/232] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:20 2022] GET /identity/v3/role_assignments?user.id=24c2ba67fef340948c7eec27a92d0a28&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=43be248131c345f4b8664cc353124638 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:21.201421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:21.202071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:21.202354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:21.202631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:21.203784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:21.205331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6c4ac-0140-4190-8d8b-32644ce6e705 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:21.207349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 115/233] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:21 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:21.214636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:21.360607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97194d8a-1a47-45c0-b13f-045c2807e4e1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:21.398312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:21.398312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:21.398312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:21.398666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/f3d95545a2b5474a84964cb441014cc1/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:21.405606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, user_id=f3d95545a2b5474a84964cb441014cc1, role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:21.407009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9465623-f8b2-4ed8-899c-34889e6a7820 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:21.497389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 119/234] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:21 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/f3d95545a2b5474a84964cb441014cc1/roles/47edb61814514a2b8e3e7b38f680c3da => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:22.759990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba039-ebad-470a-af2a-7fa31bfc56f8 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:22.760546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba039-ebad-470a-af2a-7fa31bfc56f8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:22.760546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba039-ebad-470a-af2a-7fa31bfc56f8 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:22.762108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 116/235] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:22.778680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:22.779140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:22.779212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:22.785898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:22.800961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:22.802145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:22.896271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 120/236] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:22.935291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163be88-1bec-420a-a6c3-6e596ae7840c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:22.935291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163be88-1bec-420a-a6c3-6e596ae7840c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:22.935564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:22.935638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163be88-1bec-420a-a6c3-6e596ae7840c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:22.935638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 117/237] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:22.941625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb50311-0b2d-43c4-a348-036f0637d0c8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:22.941907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb50311-0b2d-43c4-a348-036f0637d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:22.942184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb50311-0b2d-43c4-a348-036f0637d0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:22.964341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-afb50311-0b2d-43c4-a348-036f0637d0c8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:22.965577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-afb50311-0b2d-43c4-a348-036f0637d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:23.045144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 118/238] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 02:05:23.046504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e56023-d93a-474a-ad09-6433f6bb1f7b None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.046796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e56023-d93a-474a-ad09-6433f6bb1f7b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.047074 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e56023-d93a-474a-ad09-6433f6bb1f7b None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.048643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 119/239] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:23.053988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.054275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.054557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.073442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:23.074652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:23.082167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60f6ce-1765-46b8-a7b4-31a7f3df4dbc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.114250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.118019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.119682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.120323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.120601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.120905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.131589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:23.137183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:23.138069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:23.139411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 121/240] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:23.143833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.146900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 120/241] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:23.153152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.256012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d687c4-88b1-4d78-bf15-84982b76adef None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.280879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad04c7-ef85-4c61-a540-ba71a408cb0f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.293916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.294554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.294832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.295106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.296303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.297830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.299749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 122/242] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:23.300583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.318307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.318919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.319203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.319551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.320267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:23.320588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.322067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.330658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 121/243] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:23.331283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.367934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.368246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.410413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf151b4-d04b-4e48-bbe2-51cf5d2497dc None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.439174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6cbab-8b4e-4d2b-9ac3-af5b8c6f6460 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.447054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.447740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.448023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.448296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.459221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:23.464916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:23.466940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 123/244] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:23.468999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.482584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.483226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.483583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.483866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.484521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.486105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.570424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b436fd-0101-469c-bb93-d11aceab6565 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.572894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 122/245] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:23 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:23.574178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.625105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.630864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.631894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.632446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.633153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] PATH_INFO: `/v3/users/3039de202a9749169b0e6876a6f95bb5` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.650860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=3039de202a9749169b0e6876a6f95bb5)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.653022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7f437-85dc-44f0-abaa-eccdd2016ee1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.655489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 124/246] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/users/3039de202a9749169b0e6876a6f95bb5 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:23.657309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60bdb-4afe-472c-ae63-ce12117c7b96 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.657913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60bdb-4afe-472c-ae63-ce12117c7b96 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.658405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60bdb-4afe-472c-ae63-ce12117c7b96 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.660369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 125/247] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:23.667807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.667807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.667807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.689035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:23.690635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:23.703725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.715618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d6d35-ae94-46b0-b09c-c21e8c5f5a17 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.761785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.762693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.762980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.763247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.764313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:23.765069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.766496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.777503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 123/248] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:23.777912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.780098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 126/249] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:23.786050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.911166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094fc2b-c674-402e-ba70-c458420c863b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.920579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfa5cd-cbe4-4836-a088-b4587aef37c4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:23.922554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:23.964086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.965234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.965541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.965825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:23.966979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:23.968491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:23.969551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:23.970170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:23.970286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 127/250] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:23.970929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:23.971307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:23.971590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.003013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.062493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.065295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48141d37-0d51-4922-8bed-56f842f18d0f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.143488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea6723-d37e-4964-b114-0d14ad6ee896 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.194409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.194967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.195240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.195504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.207835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:24.214102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:24.214977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:24.216362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 128/251] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:24.217051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.228101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 124/252] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:24.229758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ab80-03c6-4cf7-9686-314280d530ca None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.233602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ab80-03c6-4cf7-9686-314280d530ca None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.233602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40ab80-03c6-4cf7-9686-314280d530ca None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.233602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 125/253] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:24.233602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40ab80-03c6-4cf7-9686-314280d530ca None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.262079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.308033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.340699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79031bd-6814-44ba-851f-31bc8cc7b245 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.367602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40ab80-03c6-4cf7-9686-314280d530ca None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.385139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.385773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.386306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.386585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.412193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.412941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.413294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.413659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.414405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:24.415631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.417351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.429073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 126/254] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:24.429983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.468620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.470866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29902f57-8c55-493a-b503-18bb60a0a9bd None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.533143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.548644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d636d67-ff32-43d7-ae86-9a7ee691731c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.571593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.586471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 129/255] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:24.591162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.591561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.592056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.595528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.596484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.597040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.597509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.599817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.601519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.605008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 127/256] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:24.606399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.618221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:24.619550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:24.712290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 130/257] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:24.714240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.747657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141ff56-bf94-488a-98fb-9d84885635b6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.785839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.786461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.786819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.787095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] PATH_INFO: `/v3/users/24c2ba67fef340948c7eec27a92d0a28` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.788275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] RBAC: Authorizing `identity:get_user(user_id=24c2ba67fef340948c7eec27a92d0a28)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.790149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.792787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 128/258] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/users/24c2ba67fef340948c7eec27a92d0a28 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:24.793853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.798478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.844475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df554d8c-1be9-4cde-8c0e-b5840548fec8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.850095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:24.890000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.891146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.891594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.892007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.893737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.895726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.898106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 131/259] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:24.899011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:24.908391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a613ea3-3781-4669-a96f-8b1afc4eef3d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:24.956404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:24.957679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:24.958348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:24.958787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:24.959860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=3039de202a9749169b0e6876a6f95bb5) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:24.960450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:24.962970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:24.969655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 129/260] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/role_assignments?user.id=3039de202a9749169b0e6876a6f95bb5&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=aaf25a175bb641748e1620292b72a5cb => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:24.970471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.102827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de1fe-d670-432b-bca6-c4dc6b5f8df8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.111011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f04115-efea-43d3-ac40-be8e35b582f0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.124705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:25.144274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.145093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.145394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.145685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.146883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.152477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:25.152477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 130/261] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:25.152477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.188761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.189408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.189686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.189974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.193511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:25.204443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:25.210200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:25.211159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:25.211953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 132/262] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:25.213265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.270629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549b71f-04b2-45cb-b0e2-d11e5b7f9d2c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.309413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.310027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.310310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.310585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.311322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=f3d95545a2b5474a84964cb441014cc1) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:25.311627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.313584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:25.320284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 131/263] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:25 2022] GET /identity/v3/role_assignments?user.id=f3d95545a2b5474a84964cb441014cc1&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:25.321397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.382376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:25.419862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4ee90-2d72-4b69-bd86-1b8283efbce1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.445330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb542284-b752-4092-91a4-6bab8681c0aa None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.486751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.487780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.488220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.488639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.489625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:25.490182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.491758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:25.498359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.498786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.499067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.499346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/24c2ba67fef340948c7eec27a92d0a28/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.502458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 132/264] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:25.503453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, user_id=24c2ba67fef340948c7eec27a92d0a28, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.508909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:25.511486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.582649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 133/265] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:25 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/24c2ba67fef340948c7eec27a92d0a28/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:25.666146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e4088-9071-4793-b4dd-c21d5470e404 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.681839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:25.711710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.713475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.713963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.714419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] PATH_INFO: `/v3/users/f3d95545a2b5474a84964cb441014cc1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.715746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d95545a2b5474a84964cb441014cc1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.717413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458aa93-e60e-4aa5-8ed4-89a340382275 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:25.719605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 133/266] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:25 2022] GET /identity/v3/users/f3d95545a2b5474a84964cb441014cc1 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:25.725215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:25.802645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:25.858636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b82cb-3f13-4ad1-a451-7cb7f251c8f6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:25.903648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:25.904281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:25.904562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:25.904885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:25.987011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:25.988503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:26.056526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:26.078494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f001-1e7a-4184-a06e-fa60e7d2c260 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.079028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f001-1e7a-4184-a06e-fa60e7d2c260 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.079028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f001-1e7a-4184-a06e-fa60e7d2c260 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.080383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 134/267] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:26.087579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.087920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.088171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.114944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 134/268] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:26.120790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.125714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:26.126928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:26.237193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 135/269] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:26.272095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.298072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6264dc-b5dd-4192-ad2b-2c6e6fcfe638 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.345123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.349868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.350166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.350445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.351625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:26.353808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07081ddc-4eeb-4781-aa99-3ce2c4b38da0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:26.356287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 135/270] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:26 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:26.360942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5039c9-80a1-47cb-9bf8-9238a392c2da None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.361494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5039c9-80a1-47cb-9bf8-9238a392c2da None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.362046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5039c9-80a1-47cb-9bf8-9238a392c2da None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.364342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 136/271] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:26.366176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5039c9-80a1-47cb-9bf8-9238a392c2da None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.391993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69ea88-dd3e-4bd3-8922-0fe3ae047a6a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.394730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:26.404742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:26.441865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.443224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.443982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.444881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.446520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:26.448467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af4dae-291f-4fe4-899d-f58f84a56227 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:26.451335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 136/272] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:26.452781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.453258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.453796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.474831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:26.476368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:26.499195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5039c9-80a1-47cb-9bf8-9238a392c2da None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.549462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.550218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.550508 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.550836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.551578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=f3d95545a2b5474a84964cb441014cc1) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:26.551888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:26.554128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:26.554673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 137/273] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:26.555682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.571106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 137/274] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:26 2022] GET /identity/v3/role_assignments?user.id=f3d95545a2b5474a84964cb441014cc1&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:26.588767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.704825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:26.728696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bf1de-b639-474f-9cd0-513dc0c87da5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.768540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.771622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.771622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.771622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.783247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:26.788593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:26.789732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:26.790925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 138/275] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:26.800401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:26.805030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3cd2a-04b5-4034-936e-cd95f366f4d3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.816649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:26.867703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.868368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.869136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.869136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.869661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:26.871127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:26.943981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc19f-4c21-4514-ad04-2e9cdfdd8eb1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:26.974617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 138/276] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:26 2022] POST /identity/v3/users => generated 293 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:26.995098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:26.995746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:26.996029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:26.996304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:26.997025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:26.997334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:26.998760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:27.008623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 139/277] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:27.015120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:27.157113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a9ea-e140-41d0-a2c3-d62d8068727e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:27.194541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:27.195190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:27.195471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:27.195751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] PATH_INFO: `/v3/users/202581944df647498ae667878ae8fd2c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:27.209746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] RBAC: Authorizing `identity:get_user(user_id=202581944df647498ae667878ae8fd2c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:27.211207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36be3b3-feb4-4412-9d40-b64a5eb01e92 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:27.213620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 139/278] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:27 2022] GET /identity/v3/users/202581944df647498ae667878ae8fd2c => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:27.219369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:27.232163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:27.323069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b0c3-736e-4726-8b9a-4f68a8d78854 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:27.360562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:27.361238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:27.361522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:27.361801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:27.362593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, user.id=202581944df647498ae667878ae8fd2c, scope.system=all) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:27.362912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:27.364368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c055c3-0488-499e-83ad-219785fa32e1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:27.370597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 140/279] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Aug 8 02:05:27 2022] GET /identity/v3/role_assignments?user.id=202581944df647498ae667878ae8fd2c&scope.system=all&role.id=43be248131c345f4b8664cc353124638 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:27.485860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa880-f9b0-4dcb-bdee-26ea024e9ec5 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:27.486219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa880-f9b0-4dcb-bdee-26ea024e9ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:27.486413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeaa880-f9b0-4dcb-bdee-26ea024e9ec5 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:27.487781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 140/280] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:27.493604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:27.494402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:27.495050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:27.516295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:27.517910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:27.606954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 141/281] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:27.645249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaa880-f9b0-4dcb-bdee-26ea024e9ec5 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:27.674167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:27.774443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeaa880-f9b0-4dcb-bdee-26ea024e9ec5 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:27.818519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:27.819155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:27.819432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:27.819705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:27.834271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:27.839646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:27.840515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:27.841927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 141/282] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:27.846613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:27.850548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:27.973726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667ec3f-a9bf-41a1-b097-25db471bcdcf None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.012576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.013643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.014100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.014545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.015394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:28.015868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:28.017445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:28.027298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 142/283] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:28.032424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:28.149162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:28.172873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df09bbb-2ae6-4d69-80bf-5ae2ca6be975 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.223636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.224266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.224541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.224841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] PATH_INFO: `/v3/users/3039de202a9749169b0e6876a6f95bb5` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.225976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] RBAC: Authorizing `identity:get_user(user_id=3039de202a9749169b0e6876a6f95bb5)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:28.227412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512b979-5321-4ef3-9717-43a238227146 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:28.229898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 142/284] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:28 2022] GET /identity/v3/users/3039de202a9749169b0e6876a6f95bb5 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:28.235194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:28.255982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176876c7-5d8b-4a08-b7a8-99dea984e97f None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.256282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176876c7-5d8b-4a08-b7a8-99dea984e97f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.256526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176876c7-5d8b-4a08-b7a8-99dea984e97f None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.258196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 143/285] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:28.265792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.266076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.266447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.302140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:28.303366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:28.318428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:28.400341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3f2f3-3e67-4c05-b6cb-234544cdebff None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.406385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 144/286] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:28.440531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:28.448650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.449338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.449626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.449911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.451058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:28.454016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:28.456192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 143/287] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:28 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:28.465706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:28.528817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:28.581109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e54f-0d56-45c1-89d3-f37751d5c677 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.640697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.640697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.640697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.640697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.647929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084434c-8a8f-4db4-a3b7-0a2c73b6c89d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.649530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:28.660149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:28.661161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 145/288] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:28.661936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:28.711657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:28.719604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.720274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.720554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.724783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/3039de202a9749169b0e6876a6f95bb5/roles/aaf25a175bb641748e1620292b72a5cb` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.726763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, user_id=3039de202a9749169b0e6876a6f95bb5, role_id=aaf25a175bb641748e1620292b72a5cb)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:28.728432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:28.805584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 144/289] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:05:28 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/users/3039de202a9749169b0e6876a6f95bb5/roles/aaf25a175bb641748e1620292b72a5cb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:28.812533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a193c0-2684-46d0-9d8e-708dde714806 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:28.850518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:28.851065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:28.851341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:28.851610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:28.852358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:28.852675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:28.854475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:28.860894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:28.864480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 146/290] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:28.870940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:29.007855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f113-dd3a-4629-a515-3849c514e915 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:29.047048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:29.048043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.048525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.049025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] PATH_INFO: `/v3/users/24c2ba67fef340948c7eec27a92d0a28` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:29.050409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] RBAC: Authorizing `identity:get_user(user_id=24c2ba67fef340948c7eec27a92d0a28)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:29.052047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:29.053912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 145/291] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:28 2022] GET /identity/v3/users/24c2ba67fef340948c7eec27a92d0a28 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:29.061482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:29.215629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9a238-79cc-4828-8a51-cc291a5edec5 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:29.267820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:29.268843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.269326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.269799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:29.365953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:29.369669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:29.371144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b01a2-3696-4913-a700-8a99fa1d7790 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:29.553979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 147/292] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:29.560798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:29.646557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9834b-d4bb-453a-bd7b-b08a619d4c36 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.652882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9834b-d4bb-453a-bd7b-b08a619d4c36 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.652882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9834b-d4bb-453a-bd7b-b08a619d4c36 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:29.652882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 148/293] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:29.657250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.658071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:29.659514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.659939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:29.682571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:29.684468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:29.740321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5298b30-5a3b-4999-a276-d07276c565ed None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:29.765920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 149/294] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:29.790729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:29.825425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:29.826474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.827556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.828158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:29.830613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:29.833048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:29.837277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:29.838254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 146/295] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:29 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:29.841536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:29.956321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d2da6-9d75-4450-bcf1-8d2132878f42 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:29.977380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90f9b-94ee-408c-97c1-d338dd440b1e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:29.995606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:29.996248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:29.996536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:29.996882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.005418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.008705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:30.014028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-3408e516-f883-44be-9e01-b9826403a259 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:30.014501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:30.021738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:30.022528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:30.023122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.023122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.023239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 150/296] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:30.023301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.023301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=24c2ba67fef340948c7eec27a92d0a28) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:30.023301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.023473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4324d2a-f597-45c2-8200-ad10704aae5a None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.023473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4324d2a-f597-45c2-8200-ad10704aae5a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.023473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4324d2a-f597-45c2-8200-ad10704aae5a None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.023473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 151/297] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:30.023705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.027651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6221-7557-44ac-9ae8-58d5998208e2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.027931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6221-7557-44ac-9ae8-58d5998208e2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.028205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a6221-7557-44ac-9ae8-58d5998208e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.033735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 147/298] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:29 2022] GET /identity/v3/role_assignments?user.id=24c2ba67fef340948c7eec27a92d0a28&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=43be248131c345f4b8664cc353124638 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.034780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.050260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8e8a6221-7557-44ac-9ae8-58d5998208e2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:30.051502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8e8a6221-7557-44ac-9ae8-58d5998208e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:30.136919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 152/299] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:30.138349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee63bb-f4b7-469c-8b25-16cba4229026 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.139111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee63bb-f4b7-469c-8b25-16cba4229026 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.139111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee63bb-f4b7-469c-8b25-16cba4229026 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.140587 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 153/300] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:30.145598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.145883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.146159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.154135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba805c7-5b38-47ab-b9a9-572b75657977 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.178553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:30.179782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:30.193500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.194514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.194997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.195460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.196317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:30.196991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.198731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.198847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.208231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 148/301] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.209358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.258502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 154/302] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:30.259585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.287907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.340810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35f8a3-67bf-435b-bb5d-c629e6066988 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.370913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4deb0c-0860-4958-995a-8391f881e143 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.379320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.380206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.380491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.380801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.381906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.383334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.385788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 149/303] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.390861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.412531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.413298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.413584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.413863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] PATH_INFO: `/v3/users/07ef5f645a834d1a81a3340aaeb0fc89` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.429102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] RBAC: Authorizing `identity:get_user(user_id=07ef5f645a834d1a81a3340aaeb0fc89)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.430555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.432878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 155/304] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/users/07ef5f645a834d1a81a3340aaeb0fc89 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.433680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.444529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.493797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f2ab-0664-4b7c-824e-b17971b622c1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.517646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.530011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.530633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.530909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.531171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.539255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4862cd8-c640-4d32-ab61-14c95e1d4a1d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.542350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:30.547074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:30.547943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:30.548744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:30.551979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 150/305] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:30.553160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.581405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.582018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.582296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.582566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] PATH_INFO: `/v3/groups` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.583190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.584623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.657174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 156/306] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Aug 8 02:05:30 2022] POST /identity/v3/groups => generated 229 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:30.657927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.677111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a9f0c-ca42-4732-abc7-42ef87035b8b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.728695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7048135-0466-4472-b159-8ced41372310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.729551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7048135-0466-4472-b159-8ced41372310 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.729945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7048135-0466-4472-b159-8ced41372310 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.730311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7048135-0466-4472-b159-8ced41372310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.731426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7048135-0466-4472-b159-8ced41372310 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, user.id=07ef5f645a834d1a81a3340aaeb0fc89, scope.system=all) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:30.731594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7048135-0466-4472-b159-8ced41372310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.733266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7048135-0466-4472-b159-8ced41372310 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.745011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 151/307] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/role_assignments?user.id=07ef5f645a834d1a81a3340aaeb0fc89&scope.system=all&role.id=aaf25a175bb641748e1620292b72a5cb => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.750107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.793526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff949b-0c83-4415-9cfd-bb71af250f5e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:30.835760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:30.836715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:30.837951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:30.838241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:30.838947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:30.839255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:30.840784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:30.850070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 157/308] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:30.856729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7048135-0466-4472-b159-8ced41372310 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:30.912499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:30.970222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7048135-0466-4472-b159-8ced41372310 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.010834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.011762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.012240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.012696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] PATH_INFO: `/v3/users/202581944df647498ae667878ae8fd2c` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.014002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] RBAC: Authorizing `identity:get_user(user_id=202581944df647498ae667878ae8fd2c)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:31.015704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da223-1ed1-46c2-ba88-25647e1e36a7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:31.018214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 152/309] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:30 2022] GET /identity/v3/users/202581944df647498ae667878ae8fd2c => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:31.023984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:31.059703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672142-e816-4771-ad1d-7a90d4f8ce18 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.060022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672142-e816-4771-ad1d-7a90d4f8ce18 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.060251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5672142-e816-4771-ad1d-7a90d4f8ce18 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 153/310] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:31.067683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.067963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.068240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.089190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:31.090430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:31.138076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30069682-d355-4388-b4c9-281cebda1bf2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.168812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:31.172244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 154/311] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:31.175709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.176606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.177117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.177622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] PATH_INFO: `/v3/system/users/202581944df647498ae667878ae8fd2c/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.179275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=202581944df647498ae667878ae8fd2c, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:31.181469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:31.190522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:31.241206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:31.280015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 158/312] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Aug 8 02:05:31 2022] PUT /identity/v3/system/users/202581944df647498ae667878ae8fd2c/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:31.306972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c9304-ebe7-4ae3-8bfa-153a11fd3c12 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.361431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.362103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.362382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.362652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.397776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:31.403234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:31.404156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:31.404951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 155/313] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:31.411903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:31.557945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e582b9-06b3-4b07-a240-e061763a688a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.595444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.595849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.596133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.596409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.597124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:31.597433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:31.604687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:31.608756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 159/314] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:31.613210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:31.723606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9fd9f-5da4-41b6-aa98-5d0a3106181e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.734030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:31.761125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.761760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.762035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.762311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] PATH_INFO: `/v3/users/3039de202a9749169b0e6876a6f95bb5` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:31.763435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] RBAC: Authorizing `identity:get_user(user_id=3039de202a9749169b0e6876a6f95bb5)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:31.765046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:31.766925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 156/315] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:31 2022] GET /identity/v3/users/3039de202a9749169b0e6876a6f95bb5 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:31.772458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:31.876046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:31.930406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75b356-ae13-49eb-bc25-4d2ba5543053 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:31.968853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:31.969594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:31.969958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:31.970305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.044318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:32.045823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:32.101259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:32.146448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 160/316] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:32.152690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:32.257724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594b68-dc48-4751-a4e1-569265cdf314 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:32.293719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:32.294651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.295123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.295609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.296931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:32.298547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114df8ce-3a72-421c-be6b-98077f50ee2a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:32.300634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 157/317] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:32 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:32.307392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:32.415413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bfa50-8037-4ff0-a5ca-e038d6e3b189 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:32.450968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:32.451572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.451847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.452105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.452865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d, user.id=3039de202a9749169b0e6876a6f95bb5) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:32.453170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:32.454802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3be2bf-0fb0-48d5-8b84-1d9397d08392 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:32.461072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 161/318] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:32 2022] GET /identity/v3/role_assignments?user.id=3039de202a9749169b0e6876a6f95bb5&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=aaf25a175bb641748e1620292b72a5cb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:32.944112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1380ded-a9c8-4d66-985d-148c9ad2d99e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.944513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1380ded-a9c8-4d66-985d-148c9ad2d99e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.944719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1380ded-a9c8-4d66-985d-148c9ad2d99e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.945319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc289-a71f-40d0-9016-24c4cbc0b369 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.945606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc289-a71f-40d0-9016-24c4cbc0b369 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.945881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc289-a71f-40d0-9016-24c4cbc0b369 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.946233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 158/319] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:32.947772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 162/320] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:32.951500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.951789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.952060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.952471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:32.952784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:32.953063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:32.972222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:32.972746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:32.973426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:32.973974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:32.989280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.012644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.044732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 163/321] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:33.053068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 159/322] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:33.066970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.069739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.175633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d59f0-1fc5-44ad-8888-d6aac5e8fbc9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.175963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d90c73-5556-4ce7-8d3a-cdbe813e7c9e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.212201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.212780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.213063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.213330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.214215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.214906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.215252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.215586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.216774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.218206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:33.219749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 160/323] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:33.224853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:33.225668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:33.229714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:33.230598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:33.231390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 164/324] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:33.236457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.249389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.271187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.335556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b622b2-7184-4a5a-9ffc-536d208138ca None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.348072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e8ab0-3bd2-4a20-a5d2-f390ef6fe839 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.373173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.373934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.374207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.374471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] PATH_INFO: `/v3/groups` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.375083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.376518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966cc2c-4e8e-4843-9ccd-f2f2e00c087d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:33.386624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.387234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.387518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.387793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.388464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:33.388804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.390180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:33.398326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 165/325] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:33.404288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.472945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.491480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 161/326] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Aug 8 02:05:33 2022] POST /identity/v3/groups => generated 226 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:05:33.493086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d099-d1c5-4605-b174-1631e43e8700 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.493374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d099-d1c5-4605-b174-1631e43e8700 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.493965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d099-d1c5-4605-b174-1631e43e8700 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.495378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 162/327] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:33.503350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f59e-03dc-4804-9f4f-c3f054e81f2f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.506406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.506681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.506958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.527488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:33.528802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:33.545368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.546030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.546309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.546580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] PATH_INFO: `/v3/users/07ef5f645a834d1a81a3340aaeb0fc89` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.547815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] RBAC: Authorizing `identity:get_user(user_id=07ef5f645a834d1a81a3340aaeb0fc89)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.549620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:33.551517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 166/328] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/users/07ef5f645a834d1a81a3340aaeb0fc89 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:33.558116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.614144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 163/329] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:33.638561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.672720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.688693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595ee77-c3b0-4000-8e24-3b146fc07341 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.726540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.727117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.727398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.728233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] PATH_INFO: `/v3/system/users/07ef5f645a834d1a81a3340aaeb0fc89/roles/aaf25a175bb641748e1620292b72a5cb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.729854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=07ef5f645a834d1a81a3340aaeb0fc89, role_id=aaf25a175bb641748e1620292b72a5cb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.731241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:33.756347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75f55c-c8d6-41fe-a6cb-9dc85150df3b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.796025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.796741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.797046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.797318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.800891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 167/330] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Aug 8 02:05:33 2022] PUT /identity/v3/system/users/07ef5f645a834d1a81a3340aaeb0fc89/roles/aaf25a175bb641748e1620292b72a5cb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:05:33.812157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:33.817197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:33.818168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:33.819070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 164/331] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:33.826520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:33.857423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:33.953696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3162f-2ee1-45e6-b887-df1dc43bb113 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:33.994366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:33.994987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:33.995273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:33.995545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:33.996250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:33.996508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:33.997931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:34.007211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 168/332] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:34.014421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:34.115240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:34.130587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd8aa4-4fdb-498b-9e40-ac70ec2854a1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:34.171798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:34.172442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:34.172736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:34.173012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] PATH_INFO: `/v3/users/202581944df647498ae667878ae8fd2c` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:34.174155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] RBAC: Authorizing `identity:get_user(user_id=202581944df647498ae667878ae8fd2c)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:34.175654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f9802-2eb9-4569-a050-4a90fc794445 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:34.180177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 165/333] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:34 2022] GET /identity/v3/users/202581944df647498ae667878ae8fd2c => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:34.184229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:34.258293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:34.298656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca703a9a-1064-482e-83d6-bd090a5f3961 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:34.341644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66771d2-5371-4056-8194-a540031760ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:34.342332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66771d2-5371-4056-8194-a540031760ea None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:34.342613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66771d2-5371-4056-8194-a540031760ea None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:34.342883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66771d2-5371-4056-8194-a540031760ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:34.343617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66771d2-5371-4056-8194-a540031760ea None admin] RBAC: Adding query filter params (role.id=43be248131c345f4b8664cc353124638, user.id=202581944df647498ae667878ae8fd2c, scope.system=all) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:34.343925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66771d2-5371-4056-8194-a540031760ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:34.345402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66771d2-5371-4056-8194-a540031760ea None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:34.364877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 169/334] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Aug 8 02:05:34 2022] GET /identity/v3/role_assignments?user.id=202581944df647498ae667878ae8fd2c&scope.system=all&role.id=43be248131c345f4b8664cc353124638 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:36.055891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040ae5-f1fe-4717-b770-1bf91b7b2974 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.056877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040ae5-f1fe-4717-b770-1bf91b7b2974 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.057377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b040ae5-f1fe-4717-b770-1bf91b7b2974 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.059245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 166/335] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:36.064230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b4ae4-bd3d-4c5c-a9bf-b34525e799f4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.065076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b4ae4-bd3d-4c5c-a9bf-b34525e799f4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.065550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b4ae4-bd3d-4c5c-a9bf-b34525e799f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.084644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940844a9-c4c8-4125-a499-eaeb4ab67195 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.085301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940844a9-c4c8-4125-a499-eaeb4ab67195 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.085774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940844a9-c4c8-4125-a499-eaeb4ab67195 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.088089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 167/336] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:36.093764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e68b4ae4-bd3d-4c5c-a9bf-b34525e799f4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:36.094968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e68b4ae4-bd3d-4c5c-a9bf-b34525e799f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:36.107791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4959b-b6df-4bd4-99e8-4aa6f0789c9e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.108288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4959b-b6df-4bd4-99e8-4aa6f0789c9e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.108848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4959b-b6df-4bd4-99e8-4aa6f0789c9e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.110593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 168/337] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:36.111791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.112312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.116621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.150428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:36.152076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:36.204493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 170/338] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:36.208787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.209104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.209403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.236844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:36.238276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:36.271410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 169/339] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:36.272490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:36.346956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:36.374402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 171/340] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:36.375643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:36.431640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:36.450822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b0ff4-8113-41cc-a201-7648443f32fc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:36.489295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:36.490171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.490592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.491001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.502606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:36.507548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:36.508620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 02:05:36.510034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:36.510931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 170/341] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:36.516005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:36.526042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55869f41-6e9f-4e09-9ea2-6cd3db697d69 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:36.574196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:36.574850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.575132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.575406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.597368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:36.601491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:36.602346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:36.603147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:36.611797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 172/342] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:36.612692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:36.710113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:36.734191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7c25-636b-4789-82c6-5729674854aa None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:36.807141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:36.807787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.808066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.808340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.819654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5205a2-9f47-4db5-bb87-8d677e20604d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:36.838492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:36.845279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:36.847682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:36.848477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:36.853702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 171/343] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:36.854637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:36.873043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:36.904559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:36.905204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:36.905486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:36.905760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:36.906455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:36.906806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:36.908178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:36.924657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 173/344] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:36.925665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.042280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b4d9-87d9-4fcc-872e-d87c880fcaa3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.061271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314d6a5-388c-42ac-a8d0-8049867a25b8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.071087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.087892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.088612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.090649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.090926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.091797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:37.092107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.094398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.105524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 172/345] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:37.106292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.109434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.110052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.110326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.110595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.111270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:37.111569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.112961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68dc0a-a7d1-47b4-a151-7b2615bf1d05 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.124507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 174/346] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:37.125965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc63d2-b1f7-4a1e-9295-4b1f64b92672 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.126257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc63d2-b1f7-4a1e-9295-4b1f64b92672 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.126531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc63d2-b1f7-4a1e-9295-4b1f64b92672 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.127882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 175/347] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:37.128440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc63d2-b1f7-4a1e-9295-4b1f64b92672 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.204729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.248082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a014e-93d7-40b0-8c70-d5b9dea1f392 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.259781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc63d2-b1f7-4a1e-9295-4b1f64b92672 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.295879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.296807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.297169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.297453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] PATH_INFO: `/v3/users/07ef5f645a834d1a81a3340aaeb0fc89` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.298600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] RBAC: Authorizing `identity:get_user(user_id=07ef5f645a834d1a81a3340aaeb0fc89)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.300263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.301901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 173/348] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/users/07ef5f645a834d1a81a3340aaeb0fc89 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:37.302657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.304561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.305412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.305875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.307138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.356564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.402227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.403925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6541-1b16-4918-a0eb-cb1b3bb88a3f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.429507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f71e1a-5dad-4622-849e-d6d4470840d3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.473932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.474550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.474833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.475110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.479547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.523691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 176/349] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:37.528824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.530078 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.530540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.551847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:37.554567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:37.570515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.573119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.647741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.655712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 177/350] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:37.657055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.722146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 174/351] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:37.723492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.774859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:37.807016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443af2-630d-4dee-bf1a-1a74d11b9689 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.850773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.851503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.851883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.852189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.853582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] RBAC: Adding query filter params (role.id=aaf25a175bb641748e1620292b72a5cb, user.id=07ef5f645a834d1a81a3340aaeb0fc89, scope.system=all) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:37.854068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.855609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.865483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27836fb-35d5-4894-acfc-13e944a82afe None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:37.874840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 178/352] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/role_assignments?user.id=07ef5f645a834d1a81a3340aaeb0fc89&scope.system=all&role.id=aaf25a175bb641748e1620292b72a5cb => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:37.875741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.912505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:37.913768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:37.914226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:37.914963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:37.916798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:37.919389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:37.922664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 175/353] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:37.924115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:37.966942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.024073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6856b-4097-40e3-90e8-a6c415fa5607 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.049328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d6b33-c351-4a39-bdba-ac207d68f9c3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.054527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.067507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.068177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.068460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.068762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.084350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a306b0d5-3439-4f9f-9d26-f5c12c78a399 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:38.092135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:38.093088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:38.094259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 179/354] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:38.095132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c486f18-e9a3-4c2f-88a1-0a1a9af8e9a1 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.095132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c486f18-e9a3-4c2f-88a1-0a1a9af8e9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.095132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c486f18-e9a3-4c2f-88a1-0a1a9af8e9a1 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.096474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 180/355] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:38.097195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c486f18-e9a3-4c2f-88a1-0a1a9af8e9a1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.099775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.102212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.103284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.103781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.105259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.106702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.109237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 176/356] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:37 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:38.110475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.230053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c486f18-e9a3-4c2f-88a1-0a1a9af8e9a1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.248628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0de15b-99ec-4023-bd59-8ac45fd8784a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.248869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.270357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.270997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.271277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.277260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.281424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.283233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10b8f8-9320-4409-81be-c0ebb3c95ed4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.285204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 181/357] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:38.286356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921acb3-6380-4834-86fd-8075374fdacf None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.286648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921acb3-6380-4834-86fd-8075374fdacf None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.286923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921acb3-6380-4834-86fd-8075374fdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.293903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.294979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.294979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.295144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.295772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:38.296080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.298120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.307470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2921acb3-6380-4834-86fd-8075374fdacf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:38.308718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2921acb3-6380-4834-86fd-8075374fdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:38.312146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 177/358] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:38.313023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.357691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.390949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 182/359] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:38.392207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de8a5-5e07-463d-b5bf-87945f56ca3e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.392518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de8a5-5e07-463d-b5bf-87945f56ca3e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.392901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de8a5-5e07-463d-b5bf-87945f56ca3e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.394174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 183/360] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:38.394776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de8a5-5e07-463d-b5bf-87945f56ca3e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.434823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf9963-0d77-47f6-b828-04848d75e5db None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.474038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.474682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.474961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.475235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.476341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.478123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.479716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 178/361] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:38.482149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.517232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de8a5-5e07-463d-b5bf-87945f56ca3e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.529519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.559617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.560228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.561060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.561339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.562219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=43be248131c345f4b8664cc353124638, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:38.562531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.563961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5fcb4-c5ff-4e41-8a07-2b8f592cf3ef None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.572319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 184/362] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=43be248131c345f4b8664cc353124638 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:38.573573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.573862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.574110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.596178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:38.597448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:38.608300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da99a9-f588-41fc-8bb1-2f2ff841a1ad None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.611156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.663354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.664291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.665947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.666458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.689781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 185/363] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:38.690157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.759744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:38.761346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:38.823027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee242a-6e31-4987-8ffc-25ff043bff68 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:38.826341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.872033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:38.872740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:38.873025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:38.873301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:38.885172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:38.887058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 179/364] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:38.888121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:38.890164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:38.891049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:38.891835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:38.896615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:38.896857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 186/365] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:38.897376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.016184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23df397-33bb-432e-a021-2cd34175842d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.027583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ca14b-666d-461b-99da-2c8864519d95 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.055191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.055953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.056238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.056517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.057289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:39.057601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.059051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.066777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 180/366] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:39.067684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.072605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.073310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.073593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.073868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.087961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:39.093370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:39.094288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:39.095058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:39.095970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 187/367] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:39.096213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.139536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.165343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.207094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69663e0a-41f9-4eea-9334-9d8caee19747 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.254368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.255411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.255527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b455d54-2f10-45cd-ace2-a968e7ea6543 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.256006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.256488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.258142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.260531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.262551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 181/368] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:39.264956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.303521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.304327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.304617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.304924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.308112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:39.308112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.309425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.329856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 188/369] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:39.329856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.397617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a431ae9-f13d-4f4c-a461-8d672a072120 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.439264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.440252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.440763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.441264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.442132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:39.442644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.444284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.444721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.446807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.457891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 182/370] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:39.459143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.463113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c841-a04a-4c4c-8350-051170583749 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.514693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.515334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.515619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.515898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.517752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.518660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.522656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 189/371] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:39.523528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.601330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530174c4-7cf3-4a72-a279-088e409abf20 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.650125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.652400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.652400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.652400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.659834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a6282-bc44-4f68-b4b8-3a8b91f109d4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:39.702575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:39.703544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:39.704052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:39.704523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:39.730620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.731911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.739661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.741415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.789087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:39.792164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:39.868888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 183/372] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:39.869758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.907549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 190/373] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:39.908725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:39.969801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.973933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:39.980208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6a184-29d3-4f33-a260-2a4f2d5b0a31 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.016402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.017139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.017430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.017705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.018156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c09da5-e424-4b9c-bc24-131cceb0149b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.018438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=43be248131c345f4b8664cc353124638, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:40.018747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.020193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.027057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 184/374] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=43be248131c345f4b8664cc353124638 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:40.027982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:40.056695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.057844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.058336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.058828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.060514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.063318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.065345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 191/375] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:39 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:40.072700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:40.179017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a8ac2-c50c-415e-90c9-7772f79e8ada None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.205569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f9538-f5dc-4f84-bb67-86a5d7027e97 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.221631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.222576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.223076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.223560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] PATH_INFO: `/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.225228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] RBAC: Authorizing `identity:get_project(project_id=5f307f793c6c464eafa52c1ca17d9bc6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.226881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.229726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 185/376] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:40 2022] GET /identity/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:40.231843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:40.235069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:40.246569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.246972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.247169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.247451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.248601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.250627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.256910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 192/377] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:40 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:40.259050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:40.272937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:40.363519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75ce9-2200-4bc0-980b-cdf10ad0fa49 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.382488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7775e1-521b-4f7c-b592-32819e1733e5 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.407857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.408882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.409358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.409884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] PATH_INFO: `/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.418446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] RBAC: Authorizing `identity:get_group(group_id=96c55cdfa72c49e8a79e2618c6de98bf)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.420240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.424131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 186/378] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:40 2022] GET /identity/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:40.430731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:40.430979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.431562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.431844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.432213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.433255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:40.433604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.435067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb0efb-281f-46bf-82c8-e86f99c49b62 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.448938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 193/379] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:40 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:40.543656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:40.554842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f867702-5014-48d2-9a69-67998cf02c3e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:40.594889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:40.595519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:40.595798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:40.596071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:40.596857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] RBAC: Adding query filter params (group.id=96c55cdfa72c49e8a79e2618c6de98bf, role.id=c555140ef854486a808b99fb021d4982, scope.project.id=5f307f793c6c464eafa52c1ca17d9bc6) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:40.597173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:40.598655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b207882-5cb7-4aaa-84c7-6809570c9631 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:40.605940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 187/380] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:40 2022] GET /identity/v3/role_assignments?group.id=96c55cdfa72c49e8a79e2618c6de98bf&scope.project.id=5f307f793c6c464eafa52c1ca17d9bc6&role.id=c555140ef854486a808b99fb021d4982 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:41.197812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbaa4-d666-4ebe-971d-e1dc1faf0f4c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.198207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbaa4-d666-4ebe-971d-e1dc1faf0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.198385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05dbaa4-d666-4ebe-971d-e1dc1faf0f4c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.199894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 194/381] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:41.205578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023151d3-c3fc-4b63-b3ec-333da7e355cc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.206502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023151d3-c3fc-4b63-b3ec-333da7e355cc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.207041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023151d3-c3fc-4b63-b3ec-333da7e355cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.228491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-023151d3-c3fc-4b63-b3ec-333da7e355cc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:41.230084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-023151d3-c3fc-4b63-b3ec-333da7e355cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:41.329467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 188/382] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:41.378444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dbaa4-d666-4ebe-971d-e1dc1faf0f4c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:41.407802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f783-fa28-4863-afa4-64dd663832a3 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.408148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f783-fa28-4863-afa4-64dd663832a3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.408368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8f783-fa28-4863-afa4-64dd663832a3 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.410055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 189/383] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:41.416273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.416579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.416873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.449806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:41.451021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:41.454213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:41.504314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05dbaa4-d666-4ebe-971d-e1dc1faf0f4c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:41.553192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:41.554225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.555870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.556377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.569514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:41.577554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:41.578529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:41.579367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:41.585414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 195/384] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:41.592332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:41.638352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 190/385] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:41.683550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:41.722455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:41.752899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c5aa8-9ced-477a-bbbd-6653dc0727d2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:41.771000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:41.795304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:41.796684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.797501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.798060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.799326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:41.799894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:41.801615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:41.812865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 196/386] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:41.819212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:41.899949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f840-c4a9-4fc4-bb21-debc56ede744 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:41.937782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:41.938815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:41.939307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:41.939778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:41.953281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:41.955751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c84ef-35d2-4783-833d-8a6ade5a1f53 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:41.958841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:41.959705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:41.960517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:41.963364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 191/387] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:41.969929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.007398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.008044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.008328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.009239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.010542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.012001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.013975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 197/388] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:41 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:42.019744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.077436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:42.083881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:42.106363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad17f35-614c-4a09-a949-b08e01365476 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.148961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.151965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.154294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.154817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.155769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:42.157153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.159229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.176544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfecd8-c156-4c4a-8738-56b80917b740 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.176904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 192/389] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:42.184412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.273463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.274554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.275051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.275340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.276530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.278016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.280626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 198/390] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:42 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:42.285147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.313360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920891-32db-4d23-8404-ca1cd4f330b0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.349497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.350455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.350951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.351427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.353022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.355801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.358236 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 193/391] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:42 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:42.363816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.373933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:42.472364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03047e2-c42c-42ce-a9bd-1d6950c7b341 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.503676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8650406-5685-42cb-bc56-29bee780ea04 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.512027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.512950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.513451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.513933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.515378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.517119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.519345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 194/392] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:42 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:42.524454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:42.525520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:42.562292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.562965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.563472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.563597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.569965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, group_id=c61881d8bf2b428ca6390d43e35b9a3a, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.571974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e65404-7508-4d9b-aa0f-5ad000d78acd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.634053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:42.663680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55eac5-84c9-494b-9bf7-83432299fe41 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:42.671172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 199/393] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Aug 8 02:05:42 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:05:42.705245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:42.705977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:42.706264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:42.706592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:42.708932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc4509b5ca9c436db5ef9d44f4b49d26, group_id=c61881d8bf2b428ca6390d43e35b9a3a, role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:42.710395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa3b41-6157-40f9-b739-e5b4b53e00c7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:42.783596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 195/394] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Aug 8 02:05:42 2022] PUT /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/47edb61814514a2b8e3e7b38f680c3da => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:43.100040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969d72-29cc-45a0-baa1-7862fb5420cc None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.100723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969d72-29cc-45a0-baa1-7862fb5420cc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.101213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99969d72-29cc-45a0-baa1-7862fb5420cc None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.103162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 200/395] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:43.108255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addee3-650a-4f5f-946e-87628a5fa594 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.108547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addee3-650a-4f5f-946e-87628a5fa594 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.108860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addee3-650a-4f5f-946e-87628a5fa594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.138146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a2addee3-650a-4f5f-946e-87628a5fa594 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:43.139375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a2addee3-650a-4f5f-946e-87628a5fa594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:43.164332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda953b-f77a-4c7c-bf5c-d4d4d375ad03 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.164974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda953b-f77a-4c7c-bf5c-d4d4d375ad03 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.165300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda953b-f77a-4c7c-bf5c-d4d4d375ad03 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.166879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 201/396] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:43.171984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.172272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.172548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.201691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:43.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:43.218168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 196/397] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:43.235156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4964-285e-41af-8590-1a8d777955bb None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.235473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4964-285e-41af-8590-1a8d777955bb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.235701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4964-285e-41af-8590-1a8d777955bb None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.237328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 197/398] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:43.242354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.242636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.243020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.261958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:43.263150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:43.306372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 202/399] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:43.312844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.334711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:43.349995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 198/400] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:43.351948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.455239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321529d-c5cb-4e02-9a3f-07dcd0b4956b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.456931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:43.480309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878b6f2-7267-459f-98c8-b305902ec53a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.493400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.494408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.494916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.495381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.509166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:43.515389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:43.516270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:43.517128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:43.518364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 203/401] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:43.521396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.526612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.528121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.528414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.528715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.542495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:43.548103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:43.549242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:43.550052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 199/402] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:43.553662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.617182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:43.658445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29b35-c9b0-4bfb-a819-2b2172e9f9b3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.665978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046108d-464d-4ced-b540-183cc70e4f2c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.703931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.704689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.705044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.705373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.705500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.706134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:43.706496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.706599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:43.707047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.707484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.708164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:43.717719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 200/403] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:43.721297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:43.723118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:43.726762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:43.727893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:43.728726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:43.735192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 204/404] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:43.735192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.766627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:43.849615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddf37e-be27-4414-ae4d-ee965a41cf14 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.861343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba455d-4f95-483d-bbb8-6fc0f8048d88 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:43.871900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:43.896576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.897610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.897933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:43.898330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.898561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:43.898843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:43.899725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.900510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:43.901075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:43.901577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:43.902133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:43.902727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:43.903888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:43.909134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 205/405] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:43.910106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:43.913575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 201/406] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:43.914914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.013524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.026930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e048a21-2657-4f46-8b3a-81a5d50145ff None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.049561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb78e90-9b62-4d18-9791-2a00b8dfcce5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.080234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.080779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.081067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.081345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.082037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:44.082346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.083745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.097414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 206/407] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.097414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.110480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.111536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.112384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.112888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.114441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.116486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.119866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 202/408] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:43 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.121398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.185099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.208395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6643fcf-1bf2-4052-ba12-e3110e62fa4b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.246674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.247311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.247594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.247892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.249394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.250996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.253032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 207/409] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:44 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.254006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.259931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7b3b-9e4b-472d-a5e0-89d78c3a6973 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.282980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.297228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.299067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.299067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.299067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] PATH_INFO: `/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.299647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] RBAC: Authorizing `identity:get_group(group_id=96c55cdfa72c49e8a79e2618c6de98bf)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.301105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.303086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 203/410] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:44 2022] GET /identity/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.303990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.393210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdd084-28e7-4775-81c4-4b0bd9d5f53d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.425319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778e7ac-0070-47fa-930b-e7fd4f29b88c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.440730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.441346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.442533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.443082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.443620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.446803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, group_id=c61881d8bf2b428ca6390d43e35b9a3a, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.448866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.463497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.464134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.464463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.464771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.465963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.467392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.473741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 204/411] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:44 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.473741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.537907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 208/412] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Aug 8 02:05:44 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:44.538866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.643393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281dce0-ccb1-4cc2-a917-f7fc26573beb None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.689443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.690237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.690574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.690917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] PATH_INFO: `/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.692055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] RBAC: Authorizing `identity:get_project(project_id=5f307f793c6c464eafa52c1ca17d9bc6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.693541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.695709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 205/413] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:44 2022] GET /identity/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:44.703676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:44.733970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.755079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67997e8-52cc-4c4e-ba73-bea2efeab031 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.783557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:44.796579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.797549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.797977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.802585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.802585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c670d229ecd4ff6b0387194a7b2267d, group_id=c61881d8bf2b428ca6390d43e35b9a3a, role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.802585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a36a90-9ed0-499a-a0ef-8521df0c3245 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.842652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d005978-1fd8-429b-9d48-1fc0f6bf300f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:44.884538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:44.885266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:44.885548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:44.885830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] PATH_INFO: `/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6/groups/96c55cdfa72c49e8a79e2618c6de98bf/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:44.887813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f307f793c6c464eafa52c1ca17d9bc6, group_id=96c55cdfa72c49e8a79e2618c6de98bf, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:44.889323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf37a-01c8-498e-8018-407a36375474 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:44.908599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 209/414] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Aug 8 02:05:44 2022] PUT /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d/groups/c61881d8bf2b428ca6390d43e35b9a3a/roles/47edb61814514a2b8e3e7b38f680c3da => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:05:45.033792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 206/415] 10.222.0.27 () {58 vars in 1442 bytes} [Mon Aug 8 02:05:44 2022] PUT /identity/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6/groups/96c55cdfa72c49e8a79e2618c6de98bf/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:05:45.043330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc2c1-5588-41b4-a8cb-841ec17b9286 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.043664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc2c1-5588-41b4-a8cb-841ec17b9286 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.043950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bc2c1-5588-41b4-a8cb-841ec17b9286 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.045697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 210/416] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:45.051608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.056454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.056454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.073963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:45.075204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:45.077067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48f17-5fe2-4e13-9660-13c483508e27 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.077408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48f17-5fe2-4e13-9660-13c483508e27 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.077690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a48f17-5fe2-4e13-9660-13c483508e27 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.079408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 211/417] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:45.085189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.085481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.085753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.109847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:45.111142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:45.156259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 207/418] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:45.194315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.207468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 212/419] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:45.251373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.262991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:45.334459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:45.343162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619971db-bcb6-44f6-a827-88d2d42cc1d0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.374518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211a367-4028-4ceb-891c-86d46b96a37e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.389610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.390281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.390562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.391145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.407296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:45.415589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:45.416528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:45.417385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 208/420] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:45.423122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.426159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.427173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.427797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.428280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.442574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:45.448541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:45.449457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:45.450242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:45.452912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 213/421] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:45.473922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.544389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e9bec-5d1c-4802-bfbe-6773373ce2f0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.545954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:45.583877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.584722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.585000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.585279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.586009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:45.586341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:45.592688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:45.599220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 209/422] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:45.604497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.713825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb84e2-835b-4dfa-add4-de23cb8f8e22 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.721304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cde3e-dfd7-429d-a236-02218b453ae2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.765937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.766764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.767053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.767333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.769098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:45.782677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.783285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.783562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.783832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:45.784498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:45.784836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:45.788217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:45.801975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 214/423] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:45.810159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:45.829016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:45.927402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:45.931318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:45.935826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987e955-5af1-4d61-b756-481a5592a6e8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:45.981967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:45.982576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:45.982859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:45.983136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.108616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:46.115940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 210/424] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:46.120325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.121954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.124512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.247486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:46.281057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 215/425] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:46.286260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.315631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81806d-862e-4c27-983f-5209d80f95d1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.349917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:46.350557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.350873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.351147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.360962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.362478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.364363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 211/426] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:46.371808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.408749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c2980-77a6-4d65-8e0f-970c6ed69db7 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.440511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:46.454382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:46.457665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.457665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.457665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] PATH_INFO: `/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.460149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] RBAC: Authorizing `identity:get_project(project_id=dc4509b5ca9c436db5ef9d44f4b49d26)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.463282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.468145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 216/427] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/projects/dc4509b5ca9c436db5ef9d44f4b49d26 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:46.470395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.561233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c3117-7f7c-41bd-bd8c-e9b095aabc8c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.619052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:46.637971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:46.641316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.641600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.641874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.643035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.644485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.646372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 212/428] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:46.653506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.670286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3d30-30b6-441d-9469-8bc60dee1593 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.713534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:46.714494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.715396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.715844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.717186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.718789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.720839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 217/429] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:46.727306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:46.786463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:46.831641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d6719-2448-49a6-a9c6-afb2979961f0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.872373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:46.873389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.873954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.874403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.875303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:46.875793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:46.877503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008ccdb-4506-4365-b14a-c9a925aa387b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:46.884998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 213/430] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:46.950757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8878be-d8d0-4184-bab3-7f86ebee2554 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:46.952553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149fec-6bc7-4671-905b-8f8a7604d934 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.953082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149fec-6bc7-4671-905b-8f8a7604d934 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.953753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149fec-6bc7-4671-905b-8f8a7604d934 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.955549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 214/431] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:46.961450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:46.962047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:46.962515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:46.997797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:46.999032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:47.006132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.006753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.007035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.007307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.009424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=43be248131c345f4b8664cc353124638, scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:47.010624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:47.011278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:47.025972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 218/432] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:46 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=dc4509b5ca9c436db5ef9d44f4b49d26&role.id=43be248131c345f4b8664cc353124638 => generated 611 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:47.097906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.111501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 215/433] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:47.138885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.257034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.270172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6694c8b-7b79-4f8d-a16d-3b1915b5177e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:47.310991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.311703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.311986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.312262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.324503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-28be92a8-8801-4777-a234-061e761c8216 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:47.329437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:47.330336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:47.331202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 219/434] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:47.337958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.380969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a092a-d6ee-4f31-88e0-af1570436ff7 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.381206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a092a-d6ee-4f31-88e0-af1570436ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.381595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a092a-d6ee-4f31-88e0-af1570436ff7 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.383666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 220/435] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:47.388929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.389220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.389500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.412104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:47.413330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:47.457026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b83c8-7fdc-4015-a74a-aef4b8e1b339 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:47.499825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.500449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.500777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.501057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.501733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] RBAC: Adding query filter params (name=member) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:47.502042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:47.502355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 221/436] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:47.503562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:47.516779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 216/437] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:47.519308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.530864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.602086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.639094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.668203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb672c-b195-4430-a29e-3319169427a9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:47.671635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078491d-34fd-4421-b787-c3f450afa483 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:47.709374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.709992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.710651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.710932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.712482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.713055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.713336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.713610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.724759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-decccf32-22da-4917-b904-cd81631d9080 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:47.730080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:47.730990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:47.732130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 217/438] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:47.733942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511a81-fa96-439f-a916-c69c525e49c4 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.733942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511a81-fa96-439f-a916-c69c525e49c4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.733942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511a81-fa96-439f-a916-c69c525e49c4 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.734401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 218/439] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:47.737579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79511a81-fa96-439f-a916-c69c525e49c4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.821725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:47.823211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8142b-d09f-4f3f-b877-172135f27a98 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:47.885542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79511a81-fa96-439f-a916-c69c525e49c4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:47.900384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.904506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:47.920106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:47.920737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.921022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.921298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.921966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:47.922272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:47.928166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 222/440] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:05:47.928299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:47.928374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:47.928374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:47.928374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:47.938385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 219/441] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:47.938385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:47.943286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:47.944514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:48.011831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 223/442] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:48.012691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.036757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507b216-9b91-4229-bd0c-dd8797703870 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.072064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.072709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.072993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.073266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.074391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.075834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.082050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 220/443] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:47 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.082050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.112401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edf6a5-3eaa-4c8d-a741-c037a2fa667c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.125332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.125719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.146374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.146964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.147240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.147507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.180757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c57304-6f80-4442-8b0a-3599f42f67ee None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.217798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.219247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.222592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.223216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.223490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.223760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.235194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:48.240795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:48.241914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:48.242915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 221/444] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:48.243860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.335562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 224/445] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:48.336972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.355023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.356287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15344065-6163-4fc5-8183-cef0fe8a2186 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.358591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.389089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.389790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.390067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.390347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.391451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.392931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.394637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 222/446] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.395205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.442714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f832de-664d-4dbb-a7bf-881a044c4264 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.476361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.477118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.477509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.477788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.478451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:48.478755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.480121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.488148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 225/447] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.489122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.503291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aea6bf-59ad-4d49-a96d-e733d250f2f3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.537468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.538115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.538379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.538653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] PATH_INFO: `/v3/projects/8c670d229ecd4ff6b0387194a7b2267d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.540259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] RBAC: Authorizing `identity:get_project(project_id=8c670d229ecd4ff6b0387194a7b2267d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.541673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.543304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 223/448] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/projects/8c670d229ecd4ff6b0387194a7b2267d => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.544059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.565476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.590335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.591781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e781bd-a6d7-4c18-9250-3b5ac1fbaa74 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.625083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.625691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.625975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.626984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=43be248131c345f4b8664cc353124638, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:48.627295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.628758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.635198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 226/449] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=43be248131c345f4b8664cc353124638 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.637543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.650545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e558b01-504c-4290-808f-7db4f1430a34 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.692215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.693023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.693243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.693512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.768564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.769865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03acf8-9a7c-46ed-8404-d51a33413e92 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.770092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.811368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.812013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.812731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.812731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] PATH_INFO: `/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.813777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] RBAC: Authorizing `identity:get_group(group_id=c61881d8bf2b428ca6390d43e35b9a3a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:48.815402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:48.817169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 227/450] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/groups/c61881d8bf2b428ca6390d43e35b9a3a => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:48.820512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.823632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.839503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:48.890298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 224/451] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:48.895983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:48.950896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3b229-96b8-4fc9-a5be-3902d22e8c9e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:48.998798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:48.998798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:48.998798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:48.998798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:48.999601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] RBAC: Adding query filter params (group.id=c61881d8bf2b428ca6390d43e35b9a3a, role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=8c670d229ecd4ff6b0387194a7b2267d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:49.002244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:49.002244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:49.014354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 228/452] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/role_assignments?group.id=c61881d8bf2b428ca6390d43e35b9a3a&scope.project.id=8c670d229ecd4ff6b0387194a7b2267d&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:49.020529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73a4ad-7227-4cc3-b41f-e956c792e88e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:49.059998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:49.060641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:49.060951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:49.061215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] PATH_INFO: `/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:49.062373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] RBAC: Authorizing `identity:get_project(project_id=5f307f793c6c464eafa52c1ca17d9bc6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:49.063839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:49.065591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 225/453] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:05:48 2022] GET /identity/v3/projects/5f307f793c6c464eafa52c1ca17d9bc6 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:49.072445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:49.146831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:49.181066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2be2a-7b6f-4e41-9e24-8e1dfce9b049 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:49.218705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:49.219251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:49.219525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:49.219791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] PATH_INFO: `/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:49.220926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] RBAC: Authorizing `identity:get_group(group_id=96c55cdfa72c49e8a79e2618c6de98bf)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:49.222389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef242-3421-4132-98bf-04dbdf8070a8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:49.224333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 229/454] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Aug 8 02:05:49 2022] GET /identity/v3/groups/96c55cdfa72c49e8a79e2618c6de98bf => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:49.230265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:49.241811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:49.334469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0af74b-fe62-4730-a00d-118cdc4d3222 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:49.381385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:49.381997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:49.382274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:49.382546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:49.383299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] RBAC: Adding query filter params (group.id=96c55cdfa72c49e8a79e2618c6de98bf, role.id=c555140ef854486a808b99fb021d4982, scope.project.id=5f307f793c6c464eafa52c1ca17d9bc6) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:49.383731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:49.385249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380c77-aa0b-4a10-ae1a-9dddd542700e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:49.391039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 226/455] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Aug 8 02:05:49 2022] GET /identity/v3/role_assignments?group.id=96c55cdfa72c49e8a79e2618c6de98bf&scope.project.id=5f307f793c6c464eafa52c1ca17d9bc6&role.id=c555140ef854486a808b99fb021d4982 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:51.854858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d768-7fcf-408a-b6e5-a3bca65e1897 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:51.855831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d768-7fcf-408a-b6e5-a3bca65e1897 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:51.856309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4d768-7fcf-408a-b6e5-a3bca65e1897 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:51.858214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 230/456] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:51.863019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:51.863796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:51.864258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:51.888011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:51.889547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:51.983264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 227/457] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:52.021645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4d768-7fcf-408a-b6e5-a3bca65e1897 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.182416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4d768-7fcf-408a-b6e5-a3bca65e1897 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.227298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.228001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.228307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.228578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.238262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.242791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.244301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-e106edad-f10c-43d5-90bb-85b3a058b49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:52.265357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:52.266338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:52.267166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:52.267166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.267166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:52.267884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 231/458] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:52.272709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.280701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.286903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff8c4-e996-4ab5-a4fe-371134ff38b2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.286903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff8c4-e996-4ab5-a4fe-371134ff38b2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.286903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff8c4-e996-4ab5-a4fe-371134ff38b2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.286903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 232/459] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:05:52.292375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c08639-0bf4-4b1e-9835-b446d6453ce9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.292659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c08639-0bf4-4b1e-9835-b446d6453ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.292971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c08639-0bf4-4b1e-9835-b446d6453ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.319454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-13c08639-0bf4-4b1e-9835-b446d6453ce9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:52.322570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-13c08639-0bf4-4b1e-9835-b446d6453ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:52.454583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 233/460] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:52.455787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084ffa-5850-4825-b80b-6959c04bbfc2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.456071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084ffa-5850-4825-b80b-6959c04bbfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.456339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5084ffa-5850-4825-b80b-6959c04bbfc2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.457819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 234/461] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:52.467046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.467328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.467693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.476970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f5027-d84d-48ed-9d62-e7364262a826 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.494604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:52.495840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:52.526140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.526764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.527044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.527316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.528062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:52.528369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.530046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.548399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 228/462] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:52.549377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.569984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 235/463] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:52.571120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.634995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.636351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.676527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d9fd-133c-447a-aad9-5607c75c1dc5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.689266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c299447-c32e-4ba9-b06e-f8afc1bce5a9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.714431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.715037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.715320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.715590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.725578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.726462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.726802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.727582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.728191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.728308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.731107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.731212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-9d6bd0e7-e56e-4daa-aa1a-078f09635521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:52.746079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:52.753045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 229/464] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a3dc5-a4ea-42b2-a712-3448d1b1e6ed None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a3dc5-a4ea-42b2-a712-3448d1b1e6ed None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a3dc5-a4ea-42b2-a712-3448d1b1e6ed None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 230/465] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:52.753928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a3dc5-a4ea-42b2-a712-3448d1b1e6ed None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.816645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 236/466] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:52 2022] POST /identity/v3/users => generated 246 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:52.817884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:52.874472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a3dc5-a4ea-42b2-a712-3448d1b1e6ed None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.893020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.919349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.920526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.921050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.921544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.935446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.937493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.938971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1a7bb-8153-4071-9cbb-cab0e3b55e9f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:52.947780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-4b1d04db-1939-4ae4-a4b1-5400e6a2cbf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:52.957533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:52.958503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:52.959347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:52.959347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:52.959347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:52.961864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 231/467] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:52.963653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.964180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.964733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.986532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:52.987230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:52.987518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:52.987793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:52.988457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:52.988803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:52.990203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:52.990560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:52.991990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:53.014240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 237/468] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:53.015612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.084884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 232/469] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:53.085634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.158216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cd5ea-cd8c-4120-b7ba-afc4c45f0c19 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.189337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:53.195574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63ab0e-2246-4305-bb0f-a367fd0cffcc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.206430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.207048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.207328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.207911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.208985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:53.209297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.210855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.226603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 238/470] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:53.228244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.234020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.234814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.235194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.235613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.236364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.238021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.261188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:53.308579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 233/471] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:53 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:53.316930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.365961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf6daf-4fd3-4b2d-b9f3-43360fba6051 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.416458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.417303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.417688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.418061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.436285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.438597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.455014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70e7ab-ec51-4e94-8aa2-76c4bb7ce112 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5716335e-55b7-4883-9f87-f5cbb245d02f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:53.457470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:53.458353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:53.459179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:53.459179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:53.459179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:53.464066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 239/472] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:53.465308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97824596-43cc-4def-afdb-a64bcd19c638 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.465598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97824596-43cc-4def-afdb-a64bcd19c638 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.465878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97824596-43cc-4def-afdb-a64bcd19c638 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.467759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 240/473] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:53.468738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97824596-43cc-4def-afdb-a64bcd19c638 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.496585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.497312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.497769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.498050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.499086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.501514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59220f5-38f1-439a-a74a-7d46f3d0430e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.562524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:53.589564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 234/474] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:53 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:53.589862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97824596-43cc-4def-afdb-a64bcd19c638 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.590812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.591099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.591551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.627569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:53.629432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:53.636205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:53.673057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.673690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.673972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.674245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.674919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:53.675221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.678504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.706672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 241/475] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:05:53.713611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.781840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 235/476] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:53.813328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:53.841059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64829c-da3f-4559-9fc9-e6c8a81cd16d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.880329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.880975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.881254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.881523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.882184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.884151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.894306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:53.932324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352eb32-76a2-4d5b-b8a8-7b1bc408a8d3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:53.973548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:53.974538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:53.975030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:53.975529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:53.976405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:53.978064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:53.982698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 242/477] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:53 2022] POST /identity/v3/users => generated 249 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:54.061845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.sql.core [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:05:54.061845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:05:54.061845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [parameters: {'id': 'e53ad71337944271b69ab4406c14f3ea', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:05:54.061845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result.read() Aug 08 02:05:54.083163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:05:54.084097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application yield resource Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise self.value Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application session.commit() Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:05:54.085042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:05:54.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:05:54.086840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result.read() Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:05:54.087667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [parameters: {'id': 'e53ad71337944271b69ab4406c14f3ea', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:54.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 02:05:54.089446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:54.092225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 236/478] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Aug 8 02:05:53 2022] POST /identity/v3/roles => generated 144 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:54.097989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:54.227451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a3413-2eb2-460e-b485-853a3ed276de None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:54.263260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:54.263897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:54.264176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:54.264447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:54.275988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-eb5a52db-010e-4c97-8aef-5d6b1edebead None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:54.281810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:54.283916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:54.285925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 243/479] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:05:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:54.292893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:54.294767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:54.400466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45acf-324a-4a71-96f2-19f45029f935 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:54.436972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:54.437577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:54.437851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:54.438115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:54.438774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:54.439078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:54.440580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4524f341-7d2d-42a9-9f72-ec91cbc3f06c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:54.450713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 237/480] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:55.207068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18499a-13d4-4bc6-b8f2-2fe90154d836 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.207791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18499a-13d4-4bc6-b8f2-2fe90154d836 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.207791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18499a-13d4-4bc6-b8f2-2fe90154d836 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.209274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 244/481] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:55.214408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.214888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.215039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.237951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:55.239122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:55.310725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:55.319797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 238/482] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:55.355523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18499a-13d4-4bc6-b8f2-2fe90154d836 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:55.443320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:55.501106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18499a-13d4-4bc6-b8f2-2fe90154d836 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:55.545946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:55.546584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.546864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.547138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.566392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:55.571882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:55.572798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:55.574004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 245/483] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:55.579191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:55.735787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e0c14-8173-4a2f-97c0-1dc5e92d70b5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:55.770526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:55.771055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.771331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.771591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.772251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:55.772547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:55.773971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6111867-2783-4ec6-b2cd-467de43cd1b7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:55.782690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 239/484] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:55.787813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:55.790800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1b7f-9dcd-41b4-8c4e-5ed3e39d7600 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.791316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1b7f-9dcd-41b4-8c4e-5ed3e39d7600 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.791777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd1b7f-9dcd-41b4-8c4e-5ed3e39d7600 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.793567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 240/485] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:55.798942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.799421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.799894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:55.833429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:55.836709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:55.922683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:55.934226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d2e14-e693-4866-be60-3e9da3487346 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:55.939948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 241/486] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:05:55.954045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:55.990797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:55.991692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:55.992186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:55.992629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.003579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.005333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.010814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:56.020791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:56.021657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:56.022516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:56.022516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:56.022516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:56.024266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 246/487] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:56.030136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.189331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238544d-2f0f-4fa2-9233-51bd526c0bf7 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.201644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8df8-ba6b-46df-9d4a-9f4f6202ccf4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.231100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.232128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.232406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.232690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.247482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:56.249734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.250888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.251334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.251740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.252530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-114a37c8-c8d9-42d2-b9c9-4f916172b39e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:56.252652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:56.253707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:56.254663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 242/488] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:05:56.255415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6174f-3967-4460-ae38-a3b1a75d8be7 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.255667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6174f-3967-4460-ae38-a3b1a75d8be7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.255954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6174f-3967-4460-ae38-a3b1a75d8be7 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.256444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.257440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 243/489] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:56.259576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.264940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.265224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.265506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.276861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 247/490] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:56.277944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.292946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:56.294638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:56.312153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.349585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.374293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 244/491] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:56.375518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.419501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde3de6-eaf2-47e0-8d33-c439953323f5 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.459684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.460530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.461495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.461930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.462965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:56.463404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.465042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.474129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 248/492] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:56.475161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.498941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfffe63-2237-4017-ac10-0f94bc0fa427 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.539790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.541079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.541754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.542193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.552659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.555447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:05:56.572140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:56.573193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:05:56.574143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:56.576735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 245/493] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:56.577725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.588124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.604544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bff5e7-3570-417c-8a4f-26bcc78c081e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.625865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.649187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.649987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.650269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.650555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.661220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:56.666395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:56.672920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 249/494] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:56.673858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.701215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99025a0-0e90-4e5c-a521-fd53d68474f8 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.780880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.781510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.781789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.782057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.793394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.794786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d1b6ccf8-bc95-4293-8c00-1dea5a8627ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:56.810382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:56.811286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:56.812116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:56.812116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:56.812116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:56.812116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 246/495] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:56.813049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76592-7f2b-4df1-afc5-49ce69f1dd75 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.813049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76592-7f2b-4df1-afc5-49ce69f1dd75 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.813231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76592-7f2b-4df1-afc5-49ce69f1dd75 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.819053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 247/496] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:56.819053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76592-7f2b-4df1-afc5-49ce69f1dd75 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.828478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc1724-d376-4669-9580-7c6ac3300ac0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.865784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.866395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.866671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.866941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.868202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:56.868728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.870158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.906680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 250/497] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:56.907810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:56.918769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.919015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:56.931798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76592-7f2b-4df1-afc5-49ce69f1dd75 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:56.969931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:56.970544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.970826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.971106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:56.971804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:56.972111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:56.973523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39aba0-bd8a-4fed-bd48-398fa2bd48a1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:56.986380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 248/498] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:56.986380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:56.986380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:56.986380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.002294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:57.003613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:57.020182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14225-73cb-4bcb-a64b-f949780cdbc1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.056558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.057196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.057548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.057895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.058552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:57.058868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.060249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.077307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 251/499] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:57.077307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.080017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 249/500] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:57.080823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.166142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.173824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.185341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c67c15-7a24-440e-9d88-5b5d5414f228 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.212926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2047d-162f-41f5-b878-6c1740da5f59 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.220450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.221094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.221370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.221644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.232374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.233931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.246974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.247567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.247841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.248107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:57.248259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:57.249227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:57.250094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:57.250094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:57.250094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:57.250094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 250/501] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:57.250609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.319885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.321326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9e6fa-3cfa-4c2b-8bac-8d46af2a56d0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.349526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffd2d-6d9a-47ba-87ad-241494d5fccf None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.377715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.382646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.383995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.384219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.389242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.389346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.391706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.393193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:05:57.404810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:57.410866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 251/502] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:57.419101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.428595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 252/503] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:57.430420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926de06-0809-4149-90b8-d33e4e261cc0 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.430738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926de06-0809-4149-90b8-d33e4e261cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.431060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926de06-0809-4149-90b8-d33e4e261cc0 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.436466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 253/504] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:05:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:05:57.437202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926de06-0809-4149-90b8-d33e4e261cc0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.507434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d35f6-54b5-4d31-9c14-02c865084be1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.537012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926de06-0809-4149-90b8-d33e4e261cc0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.540883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.541505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.541781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.542050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.551776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:05:57.558063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:57.559113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 252/505] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:57.559892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.577836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.578375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.578655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.578926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.579566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:57.579869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.583684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.593200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 254/506] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:57.594182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.616309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.623150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.663501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036a634-e3a5-4052-82cf-8c739b5d1ca4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.696470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8cbd10-f87f-481b-9f74-4eb5042dfdb4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.697512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.698121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.698401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.698687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.699411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:57.699715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.701204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643208d3-3514-451c-92c0-c96e13157f38 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.731250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.731875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.732156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.732432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.737051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 253/507] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:57.738238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.738524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.738796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.743295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.744768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.756401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:05:57.757619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:57.759153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:57.760162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:57.761047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:57.761047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:57.761047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:57.761047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 255/508] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:57.761456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.827563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 254/509] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:05:57.828694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.843589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.865147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133e03-8df8-46fb-9560-342cf4b231bc None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.872906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:57.899140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.899769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.900057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.900330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.901030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:57.901333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.902739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.910510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 256/510] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:57.913173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:57.931383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64ab51-e279-4fc7-b07e-a1ace099c924 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:57.965404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:57.966134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:57.966414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:57.966689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:57.973535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:57.974972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:05:57.987002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:57.987923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:57.988832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 255/511] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:57.989395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.014896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50692123-e580-4f5b-9faa-f5b853459ce8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.048230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.048853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.049133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.049405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.060225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.083347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.086167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773586e-ad5e-48b8-b0a9-cf3dc3b459a5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.116567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.118631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.121392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.121989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.122262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.122529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.123165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:58.123461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.124841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.136081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 256/512] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:58.136969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.217476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 257/513] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:58.218313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.234963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8a492-4a70-4ce7-8258-46337121bd37 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.260850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.267961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.268582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.268885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.269165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.279647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.281148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:58.295642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:58.296753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:58.297627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:58.297627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.297627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:58.297627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 257/514] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:58.297627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.311848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.323509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade1ca8-c872-400e-92ec-5e828b18ab7a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.357557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.358113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.358395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.358666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.368490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.370027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:58.385109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:58.386051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:58.386925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:58.386925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.386925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:58.386925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 258/515] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:58.387154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.400137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b781f4-0d10-4209-96cc-12b479475eb3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.432940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.433547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.433822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.434091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.434819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:58.435125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.436558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.454165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 258/516] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:58.454978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.480143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.488748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc9adc-6b7d-4351-982b-eaaafe9245f0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.522576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.523994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.524484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.524954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.525190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.532032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.533502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:05:58.545300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:58.546166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:58.547003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 259/517] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:58.547469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.556964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eaf10-97b0-4e32-b087-f5904073563d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.590788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.591415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.591695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.591961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.604245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.605754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:58.620845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:58.621806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:58.622630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:58.622630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:58.622630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:58.622630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 259/518] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:58.622630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.655760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61baa3-65c2-4067-b846-eaeb40e97efb None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.689716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.690320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.690599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.690869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.691506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:58.691809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.693231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.704509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 260/519] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:58.705406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.720442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.726943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5023b-dfae-4636-a7a5-717a680d4959 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.749498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.761635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.762238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.762511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.762780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.763459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:58.763759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.765158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.783197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 260/520] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:58.783197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.806766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b26b-5e65-4c0c-a5fe-28841cd7ca0a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.839959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.840569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.840876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.841151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.876298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba3967-611e-46cb-82bd-a89b00c8369b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:58.907994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.909502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.909958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:58.910555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:58.910831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:58.911106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:58.911962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:58.912266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:58.913693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:58.922945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:58.931863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 261/521] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:58.943051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:58.976168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.016640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 261/522] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:05:59.017491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.033153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3c78-af8d-48ee-b750-fba4347a1d21 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.067394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.067997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.068271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.068541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.069221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:59.069526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.071546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.082826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 262/523] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:59.083590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.115043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15f85-df8d-42fb-8ee6-d2fcfb8b9a69 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.128294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.148481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.149183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.149467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.149739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.150380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.151817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.185132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed481-388f-4908-80eb-f9e65b7a768d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.186682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.218492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.219113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.219390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.219660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.226898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.228430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:05:59.239708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:59.241687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 263/524] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:59.243389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.256099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 262/525] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:05:59 2022] POST /identity/v3/users => generated 247 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:05:59.256099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.359102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a3c9b-f012-4d78-846c-922b3b64b73e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.368825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738877c-11a1-4ef5-a5db-df84c43c6f0f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.403504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.404200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.404489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.404805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.405547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:59.405931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.407000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.407395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.407637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.407918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.408193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.413454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 264/526] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:59.414079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.420986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.422492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:05:59.439383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:59.440477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:05:59.442719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:05:59.442719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:05:59.442719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:05:59.442719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 263/527] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:59.442719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.455767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.468912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.555826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c05c-ba3a-44bf-b16f-1ec1a4064ea9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.591308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52714ee-1315-4a37-9237-3cddea547124 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.594349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.595064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.595415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.595760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.603901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.605515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:05:59.619380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:05:59.620237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:05:59.621298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 265/528] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:05:59.622297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.630474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.631092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.631379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.631658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.632502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:59.632908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.634872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.657925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 264/529] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:59.658702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.725911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.742018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70854baa-ef3a-49a8-a2be-a33006683dd9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.748451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:05:59.774531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789cfd-15a6-4a2e-b2f4-c91297d0f961 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.779489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.780111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.780392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.780678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.781344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:59.781663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.783025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.800308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 266/530] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:59.801233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.815067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.815919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.816448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.816890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.817829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:05:59.818279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:05:59.819979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:05:59.840033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 265/531] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:05:59.841153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:05:59.911062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04475e15-38ad-4cf2-9934-d198806c04e8 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.949106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:05:59.949855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:05:59.950236 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:05:59.950596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:05:59.958646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808ff7d-3e92-4972-a6be-b0cf45ed629d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:05:59.967876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.004635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.005723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.006489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.006912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.016020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.017690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.019823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.030066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.031713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:00.033138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:00.036258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 266/532] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:00.037103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.154577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 267/533] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:00.155692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.172698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116b7-0d09-4d59-aa59-ad0c2c6696c8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.205291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.218189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.218990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.219271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.219892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.220927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=c925987dc66a445dae3cd8eee2f316ce) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:00.221258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.222696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.230768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 267/534] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/role_assignments?user.id=c925987dc66a445dae3cd8eee2f316ce&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:00.231564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.274883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf54aac-dc1e-4bff-82d3-b7a19166ddd0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.312579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.313320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.313517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.313776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.314486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:00.314784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.316332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.318184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.330334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 268/535] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:00.338013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.368608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b16aef-4c63-4a30-90de-bd527e717e1f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.411822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.413341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.413768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.414243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.428464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.430765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:00.450841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:00.451850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:00.452714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:00.452714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:00.452714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:00.458376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 268/536] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:00.460573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.571706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9885513-b57d-487a-b10f-8e84aa2b40d3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.585401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.609707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7b60-1695-419d-b0dd-9e20b10ca305 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.632685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:00.634925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.635545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.635824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.636096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.636855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=144f1e9a377c4dd098112690cfdf24bc) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:00.637359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.638801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.648693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 269/537] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/role_assignments?user.id=144f1e9a377c4dd098112690cfdf24bc&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:00.677256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.677887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.678174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.678453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.679173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:00.679462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.681714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.704415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 269/538] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:00.710266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:00.854582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01016ba8-24fe-4702-94f9-4c7e6c0892bc None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:00.907966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:00.909037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:00.909588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:00.910057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:00.922171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:00.923949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:00.939309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:00.940155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:00.941011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:00.947919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 270/539] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:00.949900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:01.064103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021289e5-7520-4e3d-bd7f-e261950bacc9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:01.103873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:01.104500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.104805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.105077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.105784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:01.106091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:01.107546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83cda2-f158-4853-a303-fc81c86cb6a0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:01.125887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 270/540] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:01.132090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:01.178624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:01.268979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a2eb-8ebe-4a62-b2e2-0d2587273d25 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:01.312398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:01.313037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.313288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.313556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.314285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=3ce21b04744a41b590dbc8a8640ffc3a) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:01.314588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:01.316007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd82f-b20c-4b3a-95ac-2101072f5c87 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:01.324930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 271/541] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:01 2022] GET /identity/v3/role_assignments?user.id=3ce21b04744a41b590dbc8a8640ffc3a&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:01.511301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032ba6-1190-48c9-93d0-3556406b2bd6 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.512311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032ba6-1190-48c9-93d0-3556406b2bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.512827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032ba6-1190-48c9-93d0-3556406b2bd6 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.514743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 271/542] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:01.520216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114a0d-ef9e-4876-9cb2-4ae0d56eca9f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.520461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114a0d-ef9e-4876-9cb2-4ae0d56eca9f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.520723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114a0d-ef9e-4876-9cb2-4ae0d56eca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.549799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-72114a0d-ef9e-4876-9cb2-4ae0d56eca9f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:01.551034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-72114a0d-ef9e-4876-9cb2-4ae0d56eca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:01.630407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 272/543] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:01.655541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032ba6-1190-48c9-93d0-3556406b2bd6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:01.695906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:01.765081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032ba6-1190-48c9-93d0-3556406b2bd6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:01.800972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:01.801521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.801804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.802077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.806743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce070-37ca-4c59-8afa-c1805dd949c5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.807029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce070-37ca-4c59-8afa-c1805dd949c5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.807306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cce070-37ca-4c59-8afa-c1805dd949c5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.809086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 273/544] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:01.817052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:01.817520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:01.817956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:01.824766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:01.831191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:01.832249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:01.833249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:01.844219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 272/545] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:01.852787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:01.853024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:01.856708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:01.905075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:01.930272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 274/546] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 02:06:01.966576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.068631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff730538-3ed4-453a-9a86-a9173e582c71 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.083904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.137750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.138365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.138646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.138918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.139594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:02.139948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.142195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.157244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 273/547] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:02.159193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866d2ed-2fc7-4024-a9e4-609a984fb9b4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.162846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.256506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.260280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.260777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.261241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.284778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:02.290709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:02.291649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:02.292432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:02.296270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 275/548] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:02.302687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.319461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f096b96-e6fa-4909-931a-1533f8c73df3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.337024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.362011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.362647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.362930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.363203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.387506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.389012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:02.415223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:02.416202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:02.417101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:02.417101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:02.417101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:02.417101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 274/549] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:02.422831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.446977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.477042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91e4a3-7da4-4833-98f6-b88b4183a946 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.514484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.515262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.515637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.516009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.516828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:02.517234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.518878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.528477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 276/550] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:02.537617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.598643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d4c9-a57a-445f-b41f-3d3492e57a12 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.656543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.657179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.657460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.657732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.660288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:02.660288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.660288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc115e-adc8-4b2d-b9d7-961b553b02e3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.668637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.673257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 275/551] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:02.674666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e3e7-e8a5-4b3d-8ffd-1230118fd28e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.674922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e3e7-e8a5-4b3d-8ffd-1230118fd28e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.675200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748e3e7-e8a5-4b3d-8ffd-1230118fd28e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.680605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 276/552] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:02.681563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748e3e7-e8a5-4b3d-8ffd-1230118fd28e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.698308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6236-7160-4cca-9f55-ae531a782e3e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.736120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.737184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.737702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.738164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.750207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.752136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.753482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c74c6663-cfe3-4576-baa1-c9a49f9776db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:02.768919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:02.773672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 277/553] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.774530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.775562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.793814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748e3e7-e8a5-4b3d-8ffd-1230118fd28e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.796884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:02.798484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:02.831490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:02.832301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:02.832561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:02.832943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:02.840708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:02.842204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:02.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:02.857118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:02.858000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 277/554] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:02.858763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.883080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 278/555] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:02.884370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:02.965080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:02.965276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7314c1-d0b3-49a4-8fad-646dc2634465 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:02.992813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.000951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.001833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.002194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.002531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.003254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:03.003622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.005109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.014494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30e64d-9f28-4b31-8240-209483cb51d6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.022959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 278/556] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:03.023723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.051792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.052802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.053323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.053794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.055524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:03.056652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.058322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.083278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 279/557] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:03.084130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.134799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06624-bc6f-4cdd-af8c-6b7e22acc06c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.171763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.172397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.172690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.172977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.184639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:03.189646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:03.190995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:03.191802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 279/558] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:03.192496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.206235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a98f9d-37c7-47d5-b362-21030e6f5b7a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.223921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.242647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.243280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.243557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.243829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.244545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.254083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.256062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:03.269245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:03.270298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 280/559] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:03.277003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.303913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c92cb-93ec-4543-a522-380576d608c8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.340579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.341214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.341492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.341758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.386352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe02089-ad64-4ab5-8765-d1d39aca05af None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.413577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.415546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.423639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.424273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.424556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.424861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.425558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:03.425865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.427216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e3de9-dd5c-42fa-b66c-f80965d1452d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.437144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 281/560] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:03.438374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15188dfa-1b83-4a87-b6d4-b69cc011e976 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.438662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15188dfa-1b83-4a87-b6d4-b69cc011e976 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.438939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15188dfa-1b83-4a87-b6d4-b69cc011e976 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.441008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 282/561] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:03.441864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15188dfa-1b83-4a87-b6d4-b69cc011e976 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.472235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.489998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.529481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 280/562] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:03.530435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.556866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15188dfa-1b83-4a87-b6d4-b69cc011e976 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.597320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.598364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.599084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.599526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.600273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:03.600582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.602093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c57ebf-40bf-41dd-889d-98b32e12575e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.627752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 283/563] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:03.627752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.627752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.627752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.645510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:03.646628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:03.647994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361581ee-f002-47fc-b3ec-e09a324f0fa7 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.683842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.684554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.684919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.685254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.697048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.698570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.707364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:03.714054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:03.715165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:03.716076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:03.716076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.716076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:03.716076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 281/564] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:03.716354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.776689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 284/565] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:03.776689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.806135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:03.834370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384842e-115b-440e-9fe9-f2da7bc15ce4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.871853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.872480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.872790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.873069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.881899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe604107-bc36-4b47-a270-5e70cabacfb6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:03.884503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.885996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:03.900213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:03.901307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:03.902395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:03.902395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.902395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:03.905970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 282/566] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:03.905970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.917871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:03.918537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:03.918819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:03.919090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:03.928957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:03.930925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:03.944834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:03.946824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:03.947903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:03.947903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:03.947903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:03.947903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 285/567] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:03.947903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:03.962048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.010133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083cbb-d6b2-436a-88a6-e92f91339d1b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.026656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.045025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.045631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.045912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.046188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.046895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.047201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.047793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffdbc7-ddc1-4077-a347-5b8059bf649d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.048651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5792a-e6c6-4f4f-8c35-9318e1785b07 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.062052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 283/568] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.063240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bc1c2-3e5f-44b6-a68a-6e54c0093fcc None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.063522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bc1c2-3e5f-44b6-a68a-6e54c0093fcc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.063797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bc1c2-3e5f-44b6-a68a-6e54c0093fcc None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.065260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 284/569] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:04.065879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bc1c2-3e5f-44b6-a68a-6e54c0093fcc None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.082019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.082646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.082921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.083198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.093884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:04.098948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.100103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:04.100975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 286/570] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.101963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.171844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bc1c2-3e5f-44b6-a68a-6e54c0093fcc None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.186213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.199633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6e859-cc2c-4682-84bb-3b1717e12dba None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.207108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.207710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.207993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.208272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.208963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.209267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.210702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.222343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 285/571] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.227561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.234043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.234660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.234940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.235216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.235870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.236179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.237570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c2359-2e51-4e24-81cd-5718626b5cfc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.244749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.249714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 287/572] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.250918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.251207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.251482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.269379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:04.270578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:04.325713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a352ca-7f05-44f4-b7d5-d8bd8aa15d61 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.337193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 288/573] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:04.338018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.359390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.360015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.360296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.360567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.368096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.369577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:04.380732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.381808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:04.383460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 286/574] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.384050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.415297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.453043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2c661-cd05-404a-9021-d84cbbf3ef68 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.479088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.483161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95d55-83c4-49f9-a63d-c12007c17643 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.486497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.487103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.487379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.487647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.488303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.488602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.489991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.497695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 289/575] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.498460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.518308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.518936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.519216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.519488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.526699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.528139 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:04.539754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.540940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:04.541800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 287/576] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.542266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.601846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0492ea-4552-4856-ba4a-b40a01578b1d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.635541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.636169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.636447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.636747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.643750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.643885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.645253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.655847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54b2c9-cd47-47f8-b732-1d3188aac06d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:04.659067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 290/577] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.662968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.691615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19931409-4d55-4181-85c2-2d195400587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.692242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931409-4d55-4181-85c2-2d195400587f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.692621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931409-4d55-4181-85c2-2d195400587f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.692829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19931409-4d55-4181-85c2-2d195400587f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.703190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19931409-4d55-4181-85c2-2d195400587f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-19931409-4d55-4181-85c2-2d195400587f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:04.707891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.708888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:04.709689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 288/578] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.710363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19931409-4d55-4181-85c2-2d195400587f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.720758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.769588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cdce9-a5ac-4269-b452-ef813179e76a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.803413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.804014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.804309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.804573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.805414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.805718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.807163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.813224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19931409-4d55-4181-85c2-2d195400587f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.824180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 291/579] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.825085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.852676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.853330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.853668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.853946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.864120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.865581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:04.879022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:04.880034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:04.880961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:04.880961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:04.880961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:04.884777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 289/580] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:04.885544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.888931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.923221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1153e-bb5a-41e2-890a-a0f20e892109 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:04.933201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:04.957537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:04.958017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:04.958297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:04.958568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:04.959258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:04.959562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:04.961064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:04.972685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 292/581] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:04.973481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:04.984607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d6ac3-7102-430b-8acb-97c7a6c14b37 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.019359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.020051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.020404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.020782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.021478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.021885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.023351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.034891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 290/582] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.035664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.072360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54281061-44b4-4b21-b885-9651f423f272 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.098542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.105398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.106006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.106339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.106603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.107266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.107569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.108969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.117440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 293/583] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.118238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.141689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0919bc-5030-4587-834c-71e6ab81e4fe None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.149136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.176421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.177072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.177376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.177651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.188939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.190381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:05.204387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:05.210535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:05.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:05.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:05.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:05.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 291/584] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:05.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.222276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81326-a501-4f08-b91d-1c7e424bc043 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.256431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.257065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.257354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.257648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.258365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.258667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.260038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.271896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 294/585] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.272646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.306037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c8a01-4657-4693-92ef-c7dc1c3a68e6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.321739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.339201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.339876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.340156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.340426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.341255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=171d2d63e59b41a783cdbc61db788122) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.341559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.342980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.349062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 292/586] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/role_assignments?user.id=171d2d63e59b41a783cdbc61db788122&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.349737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.375013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.386628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf97584-65e5-49e0-98cc-e649b4b1c4d2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.427569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.428243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.428518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.428825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.446603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=fcfff7acf53f4d779549a9688117f05d, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.448170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.464985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd651e-f090-4487-81c6-37c7ff57641f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.497277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.497968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.498248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.498520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.510127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.511676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:05.526190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:05.527342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:05.528693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:05.528693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:05.528693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:05.531882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 293/587] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:05.532707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.543074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 295/588] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:05 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:05.544515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.589711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.696726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41fbb-67b0-4d3b-80d1-1d26865d8479 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.701397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4f131-4bd7-4638-b0af-b60720022fd9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.743158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.744054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.744483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.744934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.745452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.746149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.746432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.746705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.747352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.747654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.749847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.754524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.756151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.760380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.767296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 294/589] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.768187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:05.770716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:05.771935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:06:05.773391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:05.777990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 296/590] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:05.777990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.864557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:05.873249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a4a21-484f-4365-b5b6-cb20ded35c80 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.884303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be940e6c-380e-45f6-8aec-68062a2e8ea4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:05.911118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.911724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.911998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.912264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.912932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:05.913278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.915554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.925329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:05.926165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:05.926633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:05.927054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:05.927296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 295/591] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:05.928199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:05.935277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:05.936966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:05.950814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:05.951906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:05.952965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:05.955040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 297/592] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:05.955940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.023702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.037156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec58e30-28bb-4c69-85a2-eb35f63986a0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.070287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8bec8-4f02-4d9b-a081-fa25b5dcd5e3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.073255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.074166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.074166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.074392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.075105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:06.075411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.076892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.097573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 296/593] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:06.098577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.110957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.113102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.114065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.115767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.117506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.127714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.129347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:06.140936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:06.142104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:06:06.143272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:06.149973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 298/594] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:06.150939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.235799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf599e-73bb-46f1-9524-2a6abcfa758e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.263828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3652b-c50c-417a-8569-08b4f676615f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.274736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.275218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.275603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.275976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.276814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:06.277226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.278826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.291623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.292002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 297/595] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:06.293058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.311598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.312573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.313199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.313624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.330699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.332477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:06.352877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:06.354083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:06.355170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:06.355170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:06.355170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:06.362007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 299/596] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:06.362007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.388119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.433142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c4a16-ccf5-4a07-9c3f-b428368efa49 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.473892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ba1-7f81-4025-9fb7-c88f61ca3512 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.476634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.477528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.477965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.478418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.479330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:06.479792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.481446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.500553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 298/597] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:06.501800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.511810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.512300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.512579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.512882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/c925987dc66a445dae3cd8eee2f316ce/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.528405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=c925987dc66a445dae3cd8eee2f316ce, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.530004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.613333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 300/598] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:06 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/c925987dc66a445dae3cd8eee2f316ce/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:06.614282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.618649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfd9d8-737b-4e27-8033-3fac2fd2c9b8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.648186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.680910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:06.695636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.696498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.696811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.697092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.697772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:06.698117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.699522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.728927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 299/599] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:06.734729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.759160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae257f8-94d3-4f45-9840-555942688365 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.804331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.805259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.805712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.806136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.817170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.819064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:06.835334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:06.836998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:06.837977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:06.837977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:06.837977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:06.839686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 301/600] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:06.846251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:06.873649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5e283-6c02-442d-a5cf-0c028e90f1c3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:06.917279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:06.917921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:06.918210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:06.918479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:06.926762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:06.928368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:06.942187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:06.943257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:06.944385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 300/601] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:06.949592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.013090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.026652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.071877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bd40d-6287-444e-8887-d63f7517bc76 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.090351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515dbfb-0360-4fef-a7d0-8709616e3576 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.132542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.133195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.133476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.133746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.134825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:07.135135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.136689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.150885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 302/602] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:07.155967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.173390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.174296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.174731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.175157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.176013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:07.176483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.178117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.203987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 301/603] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:07.214467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.315341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583910e2-979d-41f4-8d97-0aebdd0b461a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.356189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.357092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.357538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.357962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.366195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.368955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:07.381606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:07.382818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:07.384016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:07.386196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 303/604] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:07.390965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.392012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.459046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ee97-22e3-4fb9-b0ad-a3095c2b6109 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.469082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.502852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.504134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.504562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.504997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/144f1e9a377c4dd098112690cfdf24bc/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.520980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703396f1-bb4d-4b40-b4fb-414d646a9c57 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.526659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=144f1e9a377c4dd098112690cfdf24bc, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.528295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1e994-b35b-464c-ae15-373034ea0ed8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.559602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.560209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.560482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.560781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.561482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:07.561778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.563329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85653415-0554-4c9e-a48d-1e4c39fab55c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.580717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 304/605] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:07.580717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f5f68-647c-4fc0-a88a-4f2279efb919 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.580717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f5f68-647c-4fc0-a88a-4f2279efb919 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.580717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f5f68-647c-4fc0-a88a-4f2279efb919 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.583774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 305/606] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:07.584485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f5f68-647c-4fc0-a88a-4f2279efb919 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.650096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.654747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 302/607] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:07 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/144f1e9a377c4dd098112690cfdf24bc/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:07.656816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.657710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.658481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.707384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:07.709920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:07.757693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f5f68-647c-4fc0-a88a-4f2279efb919 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.805086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 303/608] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:07.825255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:07.847098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.848176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.848763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.849246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/3ce21b04744a41b590dbc8a8640ffc3a/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.866419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=3ce21b04744a41b590dbc8a8640ffc3a, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:07.868876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabfd13-4be8-4b27-b3e5-b0dfc4d86fc9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:07.929507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:07.937091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 306/609] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:07 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/3ce21b04744a41b590dbc8a8640ffc3a/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:07.938342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a648c-6c4b-45dd-8db4-13a46d41e10d None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.938622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a648c-6c4b-45dd-8db4-13a46d41e10d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.943566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a648c-6c4b-45dd-8db4-13a46d41e10d None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.946620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 307/610] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:07.947184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3bbc4-3166-42fe-92b8-08b2d15b56bb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:07.963226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.963438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.963543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:07.998255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:07.998925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:07.999209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:07.999486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.000481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:08.001776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:08.017274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:08.022777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:08.023863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:08.024739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:08.027130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 304/611] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:08.033882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.146917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 308/612] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:08.173557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.213406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceffd25-bc69-4e0d-a89d-bbbc4372d30d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.221495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:08.273561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.274253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.274532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.274800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.275473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:08.275773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:08.279336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:08.303371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 305/613] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:08.310028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.329878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:08.331361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068fc3d-a7f0-4753-b71c-9a316b9ddffd None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.403646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.404329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.404608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.404898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.426201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:08.443916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:08.445202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:08.446044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 309/614] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:08.457703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.474112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a750cf-ede1-4514-ac62-fc11a808aeee None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.555973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.556615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.556916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.557187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.567593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:08.574565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b040-b7d7-4771-869b-69488289485f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.586422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:08.588027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:08.612627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:08.613836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:08.614792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:08.614792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:08.614792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:08.617207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 306/615] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:08.620400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.637975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.638642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.638922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.639197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.639880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:08.640186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:08.641600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:08.655211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 310/616] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:08.660651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.688018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:08.728989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c106b12-49c1-43ce-9625-318a127efb43 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.769236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.769851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.770131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.770402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.771055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:08.771360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:08.772747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:08.799621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 307/617] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:08.804218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.871292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc839bb4-30d6-4d00-a8ca-72333703e8bd None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:08.916447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:08.917114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:08.917397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:08.917673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:08.929830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:08.931799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:08.950455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:08.951414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:08.952291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:08.952291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:08.952291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:08.952291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 311/618] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:08.961967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:08.981084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.024834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed13bf-c45b-45e7-a0d9-b6e885b65931 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.064373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.088201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.088947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.089238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.089500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.100994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.103182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:09.125868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:09.126861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:09.127808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 308/619] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:09.133921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.139982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93634b9e-ebbb-42d6-853c-e0f8f8ec1680 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.175899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.176558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.176877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.177145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.177886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:09.178189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.180160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9de580-c0b7-4ba3-8e19-d27f44fe3668 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.205200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 312/620] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:09.206527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088d002-6cff-4079-973b-c2a86c582a35 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.206815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088d002-6cff-4079-973b-c2a86c582a35 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.207087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088d002-6cff-4079-973b-c2a86c582a35 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.208520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 313/621] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:09.213743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088d002-6cff-4079-973b-c2a86c582a35 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.274323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d7a44-3429-4a6c-8da5-eec53c04518c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.297974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.313170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.316702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.316702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.316702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.316702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:09.316702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.317051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7f16f-3724-4f94-80e1-492c9b0fab19 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.340081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 309/622] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:09.340081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.340081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.340081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.348590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088d002-6cff-4079-973b-c2a86c582a35 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.357077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:09.358315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:09.395752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.399642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.400541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.401082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.401570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.411288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.413578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:09.430641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:09.431645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:09.432584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:09.440515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 310/624] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:09.440515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.440876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 314/624] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:09.441887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.558156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273a9c1-ec28-48c8-b2e1-e855dd17c852 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.593908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad2623-16c7-482e-aa6d-8c189ea73e95 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.594336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.594973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.595248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.595514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.606102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.607538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:09.622301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:09.623313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:09.624241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:09.624241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:09.624241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:09.627961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 311/625] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:09.628791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.635102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.646294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.653375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.657105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.678393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 315/626] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:09.679923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.737900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccca4-b0e8-4625-8383-f2367ac6c4af None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.774055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.774684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.774961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.775230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.785868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:09.791424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:09.792349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:09.793237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:09.797383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 312/627] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:09.798234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.822229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd087-adfe-4f5f-92d3-7b82504a1af7 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.872445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.874115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.875533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.876368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.877351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:09.878209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:09.881402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:09.891797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:09.900075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 316/628] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:09.905239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:09.916700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b140e-2281-48a7-99dd-353de4c1876e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:09.950977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:09.951815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:09.952433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:09.953028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:09.962717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.027620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.029307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9c729-3af3-4289-a341-fc694ba053d8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.034534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812d691-ac2f-4946-9f17-fb317a740ee5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.084869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.086062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.086577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.087069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.088428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:10.088763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.090316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.098938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 317/629] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:10.100877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.118934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 313/630] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18e54f-a3a5-43de-81e7-c7be0fe8b500 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18e54f-a3a5-43de-81e7-c7be0fe8b500 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18e54f-a3a5-43de-81e7-c7be0fe8b500 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 314/631] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:10.148773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18e54f-a3a5-43de-81e7-c7be0fe8b500 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.212546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaf3b5-2a21-499b-ae72-489846a629d1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.221991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.246982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.247637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.247915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.248189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.255564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.256985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.257675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18e54f-a3a5-43de-81e7-c7be0fe8b500 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:10.269491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:10.270465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:10.271382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 318/632] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:10.274221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.296529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.297174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.297454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.297748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.308544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.310016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6236c8bf-a035-4e2e-9ce9-e83eba3b3e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:10.325519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:10.326662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:10.327564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:10.327564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:10.327564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:10.327564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 315/633] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:10.327795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.327954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.328218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.346261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:10.347385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:10.374365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.375502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad31ea-07d5-4855-8f90-8018f63fb702 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.408994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.409549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.409827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.410102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.413879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 316/634] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:10.414971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.428700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.430167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-dd9d5631-dcde-4b77-86bd-67bdb49457b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:10.444248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:10.446036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:10.447107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:10.447107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:10.447107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:10.447107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 319/635] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:10.447107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339bb5-ccf6-402d-b505-7b970f4a8e6a None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.447469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339bb5-ccf6-402d-b505-7b970f4a8e6a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.447630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97339bb5-ccf6-402d-b505-7b970f4a8e6a None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.448875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 320/636] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:10.449559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339bb5-ccf6-402d-b505-7b970f4a8e6a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.472720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.519121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6d607-a0b1-4a7d-b236-f9135d744589 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.552149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97339bb5-ccf6-402d-b505-7b970f4a8e6a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.552598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.553234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.553548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.553820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.555086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:10.555412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.556869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.568122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 317/637] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:10.569063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.588407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.589045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.589328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.589600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.590247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:10.590553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.591912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.595826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.611669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 321/638] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:10.611669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.669361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6325d-621f-4b5c-8e49-583c1c3e460b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.687606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.702841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.703466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.703747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.704018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.706396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a61b1-a1bc-403e-a5ea-5e835c9520f4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.714705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-4146f775-cc4d-404f-8f09-8d1d06754606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:10.719756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:10.720847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:10.721741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 318/639] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:10.722516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.722516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.722516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.739832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:10.741055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:10.745772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.746378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.746654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.746925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.747565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:10.747868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.749251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.760191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 322/640] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:10.760975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.811064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 319/641] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:10.811875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:10.834111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.866277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dcb8c-9db4-4c55-a5ee-8caa5b9924a1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.901792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.902454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.902741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.903019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/171d2d63e59b41a783cdbc61db788122/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.913685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:10.917895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=171d2d63e59b41a783cdbc61db788122, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.918316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b0671-7e83-45cc-9f78-0b7431ee4b41 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:10.919374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.952166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:10.952921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:10.953270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:10.953633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:10.960541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:10.961999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:10.973219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:10.974393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 320/642] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:10.975502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.048933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.072638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 323/643] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:10 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/171d2d63e59b41a783cdbc61db788122/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:11.073281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d925471-45b1-4001-a850-78bd7d58c20c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.073557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.113364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.113987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.114269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.114542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.115308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.115619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.117903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.127937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 321/644] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.128866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.187647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3025e-6f25-44f5-8fb8-2f9c011e3dab None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.223851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.224498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.224880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.225153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.233762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.235247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.247232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3b519-2cf3-4e4e-b66c-d76cfb57bd51 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:11.247414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:11.248245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:11.249395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 324/645] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:11.253406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.287779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.289848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.290138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.290437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.293820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.303515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:11.305880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:11.309806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:11.310929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 322/646] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:11.312103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.365812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f98d94-2152-4095-96f9-206526bace5c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.403258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.404113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.404562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.404857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.405599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.405906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.407340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.431040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 325/647] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.431040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.431289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b46ef2-8cbf-4448-a1c5-c411836b6045 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.465643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.466273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.466553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.466824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.477954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.479437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:11.494276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:11.495289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:11.496188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:11.496188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:11.496188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:11.500096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 323/648] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:11.500946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.543888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d7fed-1343-446f-82f3-630d36fe4378 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.554229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.554970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.580704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.581645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.582111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.582588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.583295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.583605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.585109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.604077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 326/649] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.604077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.613333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ec509-5864-4f92-a5cd-39f72c442fde None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.649171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.650148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.650629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.651079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.651918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.652401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.654036 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.662690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 324/650] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.663794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.710061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6aed5-08df-4fbf-ba62-8a5ac195a2ee None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.744978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.745603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.745885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.746158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.775434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9990f3-0d1f-45c4-ba80-cbeac8dcb8cd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.783406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.786563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:11.810985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.811601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.811883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.812155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.812835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.813144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.814555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.820559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.822150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.832246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 325/651] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.832246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.928165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e24f43-1785-4343-845e-1341e327637f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:11.928541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 327/652] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:11.929604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:11.967580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:11.968186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:11.968487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:11.968848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:11.969575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:11.969935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:11.971450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:11.988708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 326/653] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:11.988708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.018925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.041234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.058913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d959f-4a2b-4ffb-88a5-ce3cf63b45a9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.104895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.105734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.106137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.106510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.122259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.124851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.136192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4372a7-3d61-4587-aad7-2a9abe2e2c41 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:12.145653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:12.146873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:12.148593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:12.148593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:12.148593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:12.148593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 328/654] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:12.152625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.182426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.182989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.183270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.183541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.196739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.198428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:12.214451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:12.215662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 327/655] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:12.216861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.283133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf0c9-a35c-4bc2-8b31-e3219ed8ef40 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.307614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.324072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.324879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.325230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.325617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.337226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.338799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.340545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.349970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7df30-c34c-4ec8-b1ec-e4831fd63418 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:12.355554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:12.357396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:12.357396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:12.357396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:12.360749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 329/656] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:12.363802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.399286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.400441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.400915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.401788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.402893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:12.403354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.405836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.423280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 328/657] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:12.427955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.483819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a144a5-0844-4ca5-9f56-2b63ec77f94c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.529431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.530470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.531414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.531902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.533193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:12.533513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.534999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.560125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a301-13d8-40bf-bed7-e6d99016c776 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.563308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 330/658] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:12.564403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.600589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.602003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.602436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.602879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.603674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:12.604129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.605746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.610815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.623535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 329/659] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:12.623535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.640517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.695975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca18a5-442b-4274-9c6c-a60804496fd8 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.732520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.733266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.733638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.734027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.741467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.743011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.753177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9ddf-f4ff-4f31-9637-9c1f5731e5e6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:12.754895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:12.756009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 02:06:12.757805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:12.759190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 331/660] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:12.760691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:12.809189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.810082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.810791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.811709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.859214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.869963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466de33-f71b-41c0-b2f7-e71527e9c1f1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:12.901657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.903425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.908922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:12.909642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:12.909985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:12.910321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:12.920498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:12.921425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:12.922161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:12.934474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:12.935498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:12.936457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:12.940414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 332/661] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:12.941324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.020152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 330/662] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:13.021340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.051992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c46d05-d052-4c2d-8844-a35efb656562 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.088437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.089073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.089419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.089619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.090350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:13.090654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.092076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.110608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 333/663] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Aug 8 02:06:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:13.111479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.123321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.131259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c791b3-3d3a-435d-9206-0f6d9d6fe42b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.146266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.168755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.169365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.169651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.169924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.170620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:13.171426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.172903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.184542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 331/664] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:13.185370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.224554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9e1ad-d040-4d3d-97dd-a0c5ab96265b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.265954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.266590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.266871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.267146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.278863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.280197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.296902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf1d0c-a31d-4a5f-8d93-d4c8b1583f77 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-dc55c78d-7ba8-440a-8f7f-6e2cb93f7239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:13.297518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:13.298703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:13.299778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:13.299778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:13.299778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:13.299778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 334/665] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:13.305853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259a1e-bba8-4314-ab18-72b6deb0146c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.306133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259a1e-bba8-4314-ab18-72b6deb0146c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.306405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259a1e-bba8-4314-ab18-72b6deb0146c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.308324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 335/666] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:13.308896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce259a1e-bba8-4314-ab18-72b6deb0146c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.339935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.340543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.340878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.341152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.381579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.385832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.410065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.411617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d923ebb-b0cb-459e-962a-abf953ed4b6c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.426972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce259a1e-bba8-4314-ab18-72b6deb0146c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.462522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437f04-17d7-4729-a654-f375064677ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.463185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437f04-17d7-4729-a654-f375064677ec None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.463458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437f04-17d7-4729-a654-f375064677ec None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.463682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437f04-17d7-4729-a654-f375064677ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.464394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74437f04-17d7-4729-a654-f375064677ec None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=c925987dc66a445dae3cd8eee2f316ce) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:13.464809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74437f04-17d7-4729-a654-f375064677ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.466247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74437f04-17d7-4729-a654-f375064677ec None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.480708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 336/667] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/role_assignments?user.id=c925987dc66a445dae3cd8eee2f316ce&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:13.480708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437f04-17d7-4729-a654-f375064677ec None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.543688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 332/668] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:13.545383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.545845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.546189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.566809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:13.568212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:13.624185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437f04-17d7-4729-a654-f375064677ec None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.654036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.660724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 333/669] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:13.661742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.665693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.668095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.668971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.669393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.669810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.670621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:13.671071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.672842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.686942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 337/670] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:13.692285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.774178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bf26bd-7adf-4fe4-afe3-95b8e68e9221 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.810434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846d1ff-2869-4bc6-af28-12aadf7ed0fc None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:13.817548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.818186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.818466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.818739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.831378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:13.833308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:13.852197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:13.853242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:13.854146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:13.854146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:13.854146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:13.855063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:13.855984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:13.856466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:13.857008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:13.858168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 334/671] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:13.858959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.871038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:13.878486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:13.879486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:13.880332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:13.882155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 338/672] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:13.887820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:13.943658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.965059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:13.994507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabb0e6-664b-4b9b-9cf6-6fadad1cce1a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.008996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87978-4f3e-45d8-a6d0-6ad71065370f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.036422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.037079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.037408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.037691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.045517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.046971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.053098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.054840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.055329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.055763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.056563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.057034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.058552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:14.060059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:14.061186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:14.062288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:14.063826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 335/673] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:14.069141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.076282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 339/674] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.077391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.185088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fff2a3-aac4-4cc1-83e8-064013201233 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.202940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.211373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63546-685d-4ec2-a3d4-7e5e3e8325a5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.213726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.222099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.222757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.223040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.223309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.223964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.224264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.226276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.234384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 336/675] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.235255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.250456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.251072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.251352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.251624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.252325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.252630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.254137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce50b26-6a44-4845-8c30-8dc1d6216147 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.346004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8ffd-3b51-4aaa-ba01-76c8a9969826 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.387589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 340/676] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.387589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd817d-e5ee-42b9-a9ad-c18e0771d206 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.387589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd817d-e5ee-42b9-a9ad-c18e0771d206 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.387589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd817d-e5ee-42b9-a9ad-c18e0771d206 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.389451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 341/677] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:14.389890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd817d-e5ee-42b9-a9ad-c18e0771d206 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.402592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.403596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.403884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.404159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.410974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.412416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:14.424129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:14.425187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:14.426271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:14.427924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 337/678] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:14.429383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.473890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.501937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd817d-e5ee-42b9-a9ad-c18e0771d206 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.535628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f624c55-e555-45c5-81a1-430120504e2a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.537362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.537988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.538357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.538635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.548857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.550333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-019f203e-7558-4ef3-a7df-f482a57f846a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:14.565244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:14.566192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:14.567115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:14.567115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:14.567115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:14.567115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 342/679] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:14.567867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.567867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.568027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.578116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.578948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.579234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.579511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.580273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=144f1e9a377c4dd098112690cfdf24bc) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.580590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.582066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.588868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 338/680] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/role_assignments?user.id=144f1e9a377c4dd098112690cfdf24bc&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.589663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.591595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:14.593153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:14.611497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.688701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 343/681] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:14.689819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.744944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfade-ff73-4328-8572-b17abb991d7e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.746893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.792728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.793659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.793966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.794244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.795248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.795561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.797208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.810495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 339/682] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.811651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.822175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a06e27-3cd2-4fa8-8074-47411bfe2454 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.870412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.870963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.871245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.871532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.872166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:14.872472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:14.874091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:14.889930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 344/683] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:14.890119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:14.903386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:14.939839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a748-d678-40d2-8e14-fa4e5743b914 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:14.978548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:14.979128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:14.979414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:14.979684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:14.990892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:14.996844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:14.997815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:14.998656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 340/684] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:15.003815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.031844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.039417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3dfd8-d0d2-4d25-88ae-c39eeded1232 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.077720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.078332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.078723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.079016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.079744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=3ce21b04744a41b590dbc8a8640ffc3a) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:15.080052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.081529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.096810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 345/685] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/role_assignments?user.id=3ce21b04744a41b590dbc8a8640ffc3a&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:15.097829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.153146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09cd1f-3251-41fb-a34b-f6892ed7ac19 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.206373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.244488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92563d54-f72c-4be0-b366-d27a9b6eb7b4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.249622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.250276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.250562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.250839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.264896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.266459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:15.281156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:15.282134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:15.283057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:15.283602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 341/686] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:15.288865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.300497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.301960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.302470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.303505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.305289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:15.305833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.307989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.322102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 346/687] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:15.327386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.383652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.405178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab593f5-caf7-4e66-b5ab-5c3815a236b4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.442338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a18e7-bbca-494e-94d6-f23c0b1f82c9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.449356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.450151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.450529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.450900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.451749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:15.452159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.454230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.477473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 342/688] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:15.489712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.499342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.514445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.515318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.515766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.516179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.537661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.539652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:15.573211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:15.574401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:15.575396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:15.575396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:15.575396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:15.581277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 347/689] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:15.587595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.608705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94958-865d-40bb-8d04-ba1d4dd5f1c6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.651114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.655008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.655008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.655008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.666583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.732841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.734727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f2225-17c0-4ea3-bfa8-bb9091d2b8a5 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.789844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979749f-8005-4e6e-ba57-7361dfc7ba19 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:15.825230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:15.826523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.826947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.827361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.828175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:15.828762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:15.830358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e52c0-2080-40c1-95ff-79304d32ba1b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:15.844424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 348/690] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:15.850328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a350ae-e430-4833-8d05-f149aab11c71 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.851681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a350ae-e430-4833-8d05-f149aab11c71 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.852218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a350ae-e430-4833-8d05-f149aab11c71 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.854242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 349/691] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:15.855286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a350ae-e430-4833-8d05-f149aab11c71 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:15.859737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 343/692] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 02:06:15.869040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:15.869740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:15.870024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:15.895010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:15.896268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:15.911759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.937765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:15.979456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a350ae-e430-4833-8d05-f149aab11c71 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.018956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.019606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.019893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.020159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.029245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.030709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:16.044169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:16.045352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:16.046310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 350/693] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:16.046909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.103381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 344/694] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:16.104521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.156154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306854ca-5ef4-476a-8e42-21ec629af942 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.158564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.193682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.194361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.194670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.194944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.206766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.208392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.214041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f3a9b-0248-4263-931f-3715de94c4ac None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:16.223394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:16.226094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:16.228276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:16.228276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:16.228276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:16.228276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 351/695] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:16.228276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.260465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.261261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.261867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.262141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.263145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:16.263451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.265667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.292015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 345/696] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:16.292946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.328701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.341771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ae9f-d5bb-4c20-b0c8-ac7c0186ac38 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.383453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.384504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.385099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.385577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.386369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.387931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:16.400542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:16.401650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:16.402509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:16.405500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 352/697] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:16.406646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.416108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc369e-93d2-4da7-b1dc-587b9693ca47 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.431337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.463858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.464932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.465274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.465553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.466212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:16.466526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.468435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.485048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 346/698] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:16.485902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.527154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10331df7-3233-4b1f-8d5c-ded040f50fe3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.563956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.564763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.565145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.565555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.600485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.617226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c71c0c-a5a6-4dbe-bbb4-f132c3f0749a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.644700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.646090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.662168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.663287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.663872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.664551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.665632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:16.666346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.667858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.669020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.674167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 347/699] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:16.674992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.784095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 353/700] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:16.785910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.791290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c54370-c59d-4417-a251-ae93ab5ddf79 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.828164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.829086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.829468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.829919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.837804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.839384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:16.853031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:16.854166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:16.855260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:16.859123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 348/701] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:16.860122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.881216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.910150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fddbc-08ef-418c-a6ca-8437a86f3aef None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:16.934317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:16.950086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:16.950705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:16.951079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:16.951359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:16.952001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:16.952360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:16.953840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:16.961059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 354/702] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:16.961059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:16.988783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20c51-7131-4b33-acf4-7180a64f537a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.031210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.031867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.032147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.032421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.045918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.047621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a459f3bf-1415-446f-a6f9-c44e464a3c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:17.064157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:17.065195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:17.066123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:17.066123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:17.066123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:17.070406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 349/703] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:17.071915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac63de-b674-41ea-bf60-055818c98c68 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.072280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac63de-b674-41ea-bf60-055818c98c68 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.072634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac63de-b674-41ea-bf60-055818c98c68 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.074691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 350/704] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:17.075151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac63de-b674-41ea-bf60-055818c98c68 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.120416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d286c0-1755-4bae-b6da-ff4920e358b8 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.164368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.165003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.165889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.166179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.166455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.167449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:17.167762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.169397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c83b8-0af6-4323-a614-bb0add6ab38a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.188766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 355/705] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:17.188766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.188766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.188766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.209355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:17.210825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:17.212109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac63de-b674-41ea-bf60-055818c98c68 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.260352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.261386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.262302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.262772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.263642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:17.264146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.265064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.266994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.283560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 351/706] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:17.284895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.303086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 356/707] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:17.304949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.436877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78dd7db-0c3f-4d8b-98e5-5e745bc6b360 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.437471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bc30-bdce-4056-81aa-5b32e503a7e2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.464880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.475151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.484901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.484901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.484901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.485137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.485137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.485137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.485137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.485410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=171d2d63e59b41a783cdbc61db788122) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:17.485410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.485410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.485605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.485605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.488305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 352/708] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/role_assignments?user.id=171d2d63e59b41a783cdbc61db788122&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:17.489127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-48ecf9ce-af32-4f66-9e49-f45c8e561bd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:17.498301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:17.499887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:17.500820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:17.507180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 357/709] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:17.507180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc924-7d17-44b0-8bf9-6d4080b8dc2c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.507180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc924-7d17-44b0-8bf9-6d4080b8dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.507180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc924-7d17-44b0-8bf9-6d4080b8dc2c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.507180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 358/710] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:17.510881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cc924-7d17-44b0-8bf9-6d4080b8dc2c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.551578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.639677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ba59f-cac5-40d0-8ace-86f0f5331e9c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.646815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422cc924-7d17-44b0-8bf9-6d4080b8dc2c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.677556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.678193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.678516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.679683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.688123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.688768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.689042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.689309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.690007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:17.690310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.691749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2db61-5e82-4505-9518-ed15a911a5cf None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.692515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.694172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.706704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 359/711] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:17.708520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.708828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.709101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:17.711440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:17.712435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:17.713413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:17.713413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:17.713413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:17.713413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 353/712] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:17.714130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.731644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:17.732871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:17.776296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.832768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 360/713] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:17.833633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.870998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1fe52-3d1b-4aff-9f0b-f345e9977491 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:17.876884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:17.920016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:17.920886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:17.921169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:17.921831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:17.922833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:17.923150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:17.925411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:17.933447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 354/714] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:17.935305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:17.977275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83998801-c009-40dc-9542-3668c2bb52aa None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:18.046220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:18.046847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:18.047122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:18.047391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:18.048051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:18.048349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:18.051327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:18.081581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 361/715] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:18.089744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:18.139738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18af36f-591c-42f9-b0d3-699cc97f276f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:18.142693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:18.184366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:18.189445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:18.189735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:18.190055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] PATH_INFO: `/v3/services/network` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:18.190613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:18.192037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:18.192982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:18.212806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a398-9a31-4a12-8770-4cb20403b5ec None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:18.217300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:18.218280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:18.224048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 355/716] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:17 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:18.234809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:18.260760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:18.261718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:18.262220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:18.262681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:18.263630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:18.265397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:18.361676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 362/717] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:06:18 2022] POST /identity/v3/users => generated 254 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:18.371865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb914df-c290-4751-a311-aab966ee5d88 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:18.424079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:18.424631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:18.424939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:18.425213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:18.425854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] RBAC: Adding query filter params (name=network) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:18.426158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:18.427524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85038f11-9cf7-405d-8f3d-c651330f1dbb None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:18.437115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 356/718] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:18.439263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:18.601000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82408e-db69-4e53-a943-f58a587ab79b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:18.634655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:18.635276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:18.635556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:18.635830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:18.636476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:18.636824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:18.638201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389739b-9bba-4e3a-beb2-f02b904e15a9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:18.643490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 363/719] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:19.246896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd27bda-2cde-4d76-95fa-315e5aa6159a None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:19.247408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd27bda-2cde-4d76-95fa-315e5aa6159a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:19.247801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd27bda-2cde-4d76-95fa-315e5aa6159a None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:19.249532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 357/720] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:19.254980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3f44-47f7-46d0-af7f-b68eab37027a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:19.255434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3f44-47f7-46d0-af7f-b68eab37027a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:19.255718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b3f44-47f7-46d0-af7f-b68eab37027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:19.263856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:19.285243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a29b3f44-47f7-46d0-af7f-b68eab37027a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:19.286525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a29b3f44-47f7-46d0-af7f-b68eab37027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:19.373853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 364/721] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:19.401684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd27bda-2cde-4d76-95fa-315e5aa6159a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:19.448724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:19.542464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd27bda-2cde-4d76-95fa-315e5aa6159a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:19.592119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:19.592762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:19.595644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:19.595644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:19.595644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:19.595644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc7b43-7793-4e5e-938c-a9f243a6e27a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:19.714593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 358/722] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:19 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:19.837545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd17635-98e9-482e-8946-32307fd4e795 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:19.837846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd17635-98e9-482e-8946-32307fd4e795 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:19.838027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd17635-98e9-482e-8946-32307fd4e795 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:19.839661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 365/723] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:19.845301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:19.845585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:19.845860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:19.873806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:19.875103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:19.963857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 359/724] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:19.988606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd17635-98e9-482e-8946-32307fd4e795 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.104154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd17635-98e9-482e-8946-32307fd4e795 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:20.140273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:20.140942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.141225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.141559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.143409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:20.153793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-be27e851-71ce-40b1-9d04-e5fb11c4fe0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:20.158733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:20.159672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:20.160635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 366/725] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:20.166609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.186018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:20.296570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb656122-7e87-4a9f-befa-f641085997e3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:20.343815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:20.344789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.345307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.345787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.346841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:20.347349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:20.349000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:20.358275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592cd1f-579f-482a-90e1-4b80461631f3 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.359198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592cd1f-579f-482a-90e1-4b80461631f3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.359731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592cd1f-579f-482a-90e1-4b80461631f3 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.362973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 367/726] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:20.368771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.369297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.369751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.379446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 360/727] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:20.383712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.401900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:20.403423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:20.537076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:20.543386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 368/728] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:20.565797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e92dd-17d1-4e73-bc04-5d1933836930 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:20.567435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.646857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:20.648393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.649045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.649483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.660771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:20.662350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:20.679851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:20.681105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:20.682218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:20.682218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:20.682218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:20.683635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 361/729] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:20.688809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.715927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526c1b-32ad-4b7b-ac3d-a2fbccdea21f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:20.757744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:20.770702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:20.772029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.773211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.773720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.787278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:20.793774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:20.794714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:20.795531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:20.798008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 369/730] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:20.809022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.874840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660b7fd-6ecf-4302-97cb-c2371e0da277 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:20.937039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:20.947475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:20.948087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:20.948816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:20.948816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:20.949304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:20.949605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:20.951959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:20.977928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 362/731] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:20.982663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:20.998638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15664edf-bb2f-473a-a5c8-7c9dcaf43a27 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.064137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.064773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.065049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.065317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.065985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:21.066286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.067849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cd571-5f16-4164-8600-6d0597326939 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.078461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 370/732] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:21.082695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4bb2-b68f-4b2b-b2ec-ed1607905b46 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.083195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4bb2-b68f-4b2b-b2ec-ed1607905b46 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.084626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4bb2-b68f-4b2b-b2ec-ed1607905b46 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.091667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 371/733] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:21.091667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4bb2-b68f-4b2b-b2ec-ed1607905b46 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.141444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd84a99-8909-445b-824b-a3082a714a2e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.157388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.177000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.177627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.181152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.181152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.186293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.187837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-9ca0a825-665b-447b-89f1-420348b8533e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:21.200985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:21.201977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:21.202914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 363/734] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:21.208554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.208554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.208554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.224804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:21.226339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:21.243258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4bb2-b68f-4b2b-b2ec-ed1607905b46 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.278925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.293201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.295322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.295322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.295322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.299556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 364/735] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:21.300786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.320468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.322940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:21.340995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:21.341959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:21.342839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:21.342839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:21.342839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:21.349162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 372/736] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:21.350411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.442536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41757c1-74d9-42c6-9446-52740f34f284 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.457534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.462325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39aac6-4a5b-4ec2-a8fd-2a75785a78f5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.483246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.483856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.484132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.484403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.485585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:21.485899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.487801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.500056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.500701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.500991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.501260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.513510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 365/737] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:21.513510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.513768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-27705d4e-3c3d-4bb2-89e4-2dbfdbb57c3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:21.516619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:21.517585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:21.518419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:21.522544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 373/738] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:21.523805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc81f-a5bc-4b78-b493-90259277dab4 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.524087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc81f-a5bc-4b78-b493-90259277dab4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.524361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc81f-a5bc-4b78-b493-90259277dab4 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.525642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 374/739] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:21.526323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc81f-a5bc-4b78-b493-90259277dab4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.561539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.625595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cf0b-fce0-4fe5-ab3c-6ac455beefd6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.645053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc81f-a5bc-4b78-b493-90259277dab4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.661048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.661658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.661937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.662208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.663375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:21.663706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.665361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.681548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 366/740] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:21.681548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.692381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.693134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.693361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.693682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.723233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.768654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.770129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.788468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.791641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e3faf-2ef2-41dd-a002-0f5a7b65faad None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:21.829301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:21.829926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.830208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.830483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.831265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:21.831677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:21.833063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d93c2-a10e-4ab7-b682-33600e0b5efa None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:21.846549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 367/741] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:21.846549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39059-283f-4758-a441-8da984b8d918 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:21.846549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39059-283f-4758-a441-8da984b8d918 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:21.846549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39059-283f-4758-a441-8da984b8d918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:21.863606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7df39059-283f-4758-a441-8da984b8d918 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:21.864836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7df39059-283f-4758-a441-8da984b8d918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:21.874896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 375/742] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 02:06:21.875723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.936095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:21.940991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 368/743] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 02:06:21.941903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39059-283f-4758-a441-8da984b8d918 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:21.993247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fc3f2-f9f8-41ba-9d62-391823718979 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.029951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.030584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.030864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.031136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.039999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.041465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.049711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:22.055321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.056231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:22.062409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 376/744] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.063361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39059-283f-4758-a441-8da984b8d918 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.063460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.098440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.099166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.099454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.100439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.110499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.112153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:22.127702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.128902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:22.129802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:22.129802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.129802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:22.129802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 369/745] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.129802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.174247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13121e5f-a8a8-49e9-a4f1-c43aaa447083 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.192886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.211570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.212198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.212476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.212777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.224928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.227053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:22.242403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.243391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:22.245009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:22.245009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.245009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:22.245179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0acdaa-e7d7-4745-ad47-dd250fe6353b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.245252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 377/746] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.245252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.281690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.282594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.283069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.283513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.284343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.286033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.352696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 370/747] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:22 2022] POST /identity/v3/services => generated 240 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:22.352696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.353036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab05d96-f525-48ac-b6f8-a37db18e5acb None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.356822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.400881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.401509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.401787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.402058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.402807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:22.403111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.404568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.428353 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 378/748] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:22.430110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.462273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.471914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe66f14-3340-4ec8-a61e-f34fac7f4154 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.510256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.511065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.511065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.511321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.512411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:22.512411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.513757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb36a64-fd5f-49df-86a9-b2083a816944 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.526304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 371/749] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:22.528117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e044e1-0bd0-4c28-80a2-c57312991fe4 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.528401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e044e1-0bd0-4c28-80a2-c57312991fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.528692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e044e1-0bd0-4c28-80a2-c57312991fe4 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.530005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 372/750] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:22.530622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e044e1-0bd0-4c28-80a2-c57312991fe4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.550339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208d8b-dfd6-4b10-9b33-93a1641704c0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.591113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.592170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.592447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.595756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.595756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:22.595756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.596221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.601992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.609081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 379/751] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:22.609725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.649987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e044e1-0bd0-4c28-80a2-c57312991fe4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.690741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.691379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.691661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.691935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.702759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.703737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.704216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b0608fb3-fb20-4d87-9b3f-a27abb2787fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:22.719562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.720584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:22.721485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:22.721485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:22.721485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:22.721664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e002e-0af7-4a3b-9886-9bcd5d67af8c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.725702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 373/752] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.726728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.727014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.727291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.745834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:22.747035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:22.758331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28009f-ab61-471f-9988-329bba64799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.758969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28009f-ab61-471f-9988-329bba64799f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.759252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28009f-ab61-471f-9988-329bba64799f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.759526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28009f-ab61-471f-9988-329bba64799f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.770039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28009f-ab61-471f-9988-329bba64799f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.774622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28009f-ab61-471f-9988-329bba64799f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-da28009f-ab61-471f-9988-329bba64799f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:22.787091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.789306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 380/753] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.790945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28009f-ab61-471f-9988-329bba64799f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.832687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 374/754] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:22.832687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.883618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.901478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28009f-ab61-471f-9988-329bba64799f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.936414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.937074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.937358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.937633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.938791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b5b47-ea9b-44af-8ccd-c0a61035cad6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:22.944849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.946391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:22.957417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:22.958290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:22.959149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:22.964025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 381/755] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:22.964909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:22.968327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:22.976721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:22.977375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:22.977649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:22.977921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:22.980873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:22.980873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:22.980873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:22.991970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 375/756] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:22.992998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.076426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55ce8-6594-46c5-be79-2e278ecfa803 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.104059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f781c92-f1db-4687-b039-4b37f18375f8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.111744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.111942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.112361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.112643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.112956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.113692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.114003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.115473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.136820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 382/757] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.137972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.145935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.146586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.146871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.147150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.147748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.149155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:23.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:23.159462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:23.160283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 376/758] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Aug 8 02:06:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:23.161153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.213203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.254347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821d6e-a406-44a6-bd6b-a155e3c41aec None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.267579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8cc4-b04e-4a9a-82e0-941d0dc62862 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.294829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.295460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.295703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.295967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.296788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.297199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.298612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.301760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.302388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.302674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.302949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.309953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.315751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 383/759] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.315875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.315975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:23.327493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:23.328384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 377/760] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:23.329340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.361229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.431264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004d628-3285-4266-a03a-b9e1dfe91df9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.435049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118b22e-4339-4068-b91b-7543f6d92b7e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.457843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.470478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.471182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.471542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.471885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.472941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.473555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.473831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.474096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.474735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.475035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.476520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.486886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 378/761] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.487926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.546912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.548377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.589898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.596956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47153e5a-b4f7-41c4-9d02-88dc7017ef3f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.635692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.636346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.636556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.636862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.637659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=171d2d63e59b41a783cdbc61db788122) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.637899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.639389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.645615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 379/762] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/role_assignments?user.id=171d2d63e59b41a783cdbc61db788122&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.646325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.671243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 384/763] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:23.672185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.699595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.766369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390341d-0837-4ee1-b094-c502cb8b48b0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.785077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68718a85-992a-4425-8836-6e115c1a8430 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.811897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.812930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.813416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.813923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.814992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.815509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.817262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.827863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.828488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.828800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.829083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:23.829734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:23.830044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:23.831423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:23.834035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 380/764] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.835749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.837809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 385/765] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:23.844778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:23.856833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.946911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185dd01-68f6-4c90-8d71-3574e55fe3fd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.958475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:23.978364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3602d30-501c-405e-b242-a773c8ffe20a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:23.989977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:23.990631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:23.991896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:23.992243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.009910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.011783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.025351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.026305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.026797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.027271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.028277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] RBAC: Adding query filter params (interface=public, service_id=0565c1275ded43d08ca2232e40aa1b4c, region_id=RegionOne) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:24.028864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.030493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:24.034423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:24.035454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:24.036345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:24.036345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:24.036345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:24.040819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 381/766] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:24.042118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.050274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 386/767] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:23 2022] GET /identity/v3/endpoints?service_id=0565c1275ded43d08ca2232e40aa1b4c&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:24.051523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.120023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:24.157385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7d6a1-db05-4537-8ba3-375f2736530c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.184498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb982d81-4889-4411-8f32-0ca9b31e2eed None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.196424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.197067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.197347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.197622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.199642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=fcfff7acf53f4d779549a9688117f05d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.201246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.221729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.224244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.224244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.224244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.224244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:24.224244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.226088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.238452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 387/768] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:24.240776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.254768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:24.292563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 382/769] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:24 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:24.293841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.398414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c69fb6-6bce-40c3-a782-dfd36218be88 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.458384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.459010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.459289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.459565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.460244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.461861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c6ead-8ad2-4de3-9f56-0dd7fd975499 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.462857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496737e9-2ca6-4782-b4d2-d758baa740a1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.469591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 388/770] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:24.470332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed92b5-9cc6-4183-8a6f-47022dd0a012 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.470614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed92b5-9cc6-4183-8a6f-47022dd0a012 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.471156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed92b5-9cc6-4183-8a6f-47022dd0a012 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.472792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 389/771] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:24.480465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.480948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.481572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.503119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:24.509338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:24.510477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.510606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:24.511119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.515598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.515598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.524397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.526722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:24.543546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:24.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:24.547820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 383/772] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:24.551834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.616583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 390/773] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:24.657506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.658656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:24.696782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4903c-0d1c-4136-a87e-2193b1479e62 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.749398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.750041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.750321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.750596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.751299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:24.751608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.753072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.766106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 384/774] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:24.772002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.781129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e8924-b5f1-48a1-8d6c-7bca356e7a44 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.823091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.823727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.824005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.824274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.824963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.826376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:24.836241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:24.837320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:24.838176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:24.840789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 391/775] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:24.843472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:24.853066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:24.916075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c52e5-5608-4efa-af44-934f2237ea20 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:24.965329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:24.966863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:24.967147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:24.968757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:24.970996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=76ca42843b2941e0860bca74f5d7c5f6) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:24.971333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:24.974729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:24.988481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 385/776] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/role_assignments?user.id=76ca42843b2941e0860bca74f5d7c5f6&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:24.990345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:25.032373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93f904-b402-41a5-a2a0-978b43c53cb1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:25.076485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:25.077152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:25.077433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:25.077709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:25.078365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] RBAC: Adding query filter params (name=network) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:25.078672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:25.080096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:25.091315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 392/777] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:25.092833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:25.227586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c348-d132-4a75-8045-78ead804cbe1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:25.242181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:25.263022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:25.263636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:25.263919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:25.264195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:25.264937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] RBAC: Adding query filter params (type=network) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:25.265249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:25.266619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:25.276803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 386/778] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:25.278971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:25.387217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e4571-1047-41e2-82a0-adc7bd712b7f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:25.419003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:25.424212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:25.424867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:25.425149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:25.425425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:25.426136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] RBAC: Adding query filter params (interface=public, service_id=60ebc198e2ea455cb696dbf84dc18b12, region_id=RegionOne) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:25.426450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:25.427861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfaec30-68ec-448e-b8b3-33b171034d5f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:25.433610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 393/779] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:25 2022] GET /identity/v3/endpoints?service_id=60ebc198e2ea455cb696dbf84dc18b12&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:25.447474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:25.546712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1116-66f2-4d35-8288-b439b45d6f6a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:25.581741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:25.582364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:25.582649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:25.582937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:25.583554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:25.585577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092136de-04a7-4374-bc61-661fdb6610e3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:25.595937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 387/780] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:25 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:26.251821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61b10-cd66-4c00-be52-66701e6257a9 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.252346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61b10-cd66-4c00-be52-66701e6257a9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.252449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed61b10-cd66-4c00-be52-66701e6257a9 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:26.254107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 394/781] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:26.259877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.260279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.260446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:26.285678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:26.286949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:26.330926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:26.371790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 388/782] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:26.406658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61b10-cd66-4c00-be52-66701e6257a9 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:26.538623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed61b10-cd66-4c00-be52-66701e6257a9 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:26.584148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:26.585396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.585663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.585901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:26.602829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:26.607649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:26.608789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:26.609842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:26.612152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 395/783] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:26.618562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:26.739816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3e6cf-1cba-4347-b49a-78ac00ebbd3e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:26.793523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:26.794061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.794340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.794611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:26.795276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:26.795574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:26.796968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3685db4-d0bb-4a76-b5e6-c85e0d075b04 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:26.806488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 389/784] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:26.811758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:26.892486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:26.967747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0358e-6fe3-4628-9009-5efb7fdfd5ea None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:26.973506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48780-4d70-4e8d-8d87-da0db497b942 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.973787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48780-4d70-4e8d-8d87-da0db497b942 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.974064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48780-4d70-4e8d-8d87-da0db497b942 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:26.980777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 390/785] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:26.981929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25531e-8be2-461b-b595-2e09b7b24113 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:26.984691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25531e-8be2-461b-b595-2e09b7b24113 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:26.984691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25531e-8be2-461b-b595-2e09b7b24113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.016813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8d25531e-8be2-461b-b595-2e09b7b24113 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:27.018697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8d25531e-8be2-461b-b595-2e09b7b24113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:27.021396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.022202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.022478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.022747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.039712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.041376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.055413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8144e93c-36c7-4c7f-a298-4374de9cbe6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:27.057007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:27.057918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:27.059386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:27.059386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:27.059386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:27.059386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 396/786] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:27.059724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2f99b-c901-43af-bfcb-02ed3ad76583 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.059981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2f99b-c901-43af-bfcb-02ed3ad76583 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.060248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2f99b-c901-43af-bfcb-02ed3ad76583 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.062830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 397/787] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:27.066517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2f99b-c901-43af-bfcb-02ed3ad76583 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.112434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 391/788] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:27.113782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.114064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.114331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.133433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:27.134618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:27.191430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2f99b-c901-43af-bfcb-02ed3ad76583 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.202290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.208974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 392/789] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:27.217464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.229018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.229860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.230288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.230705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.231496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:27.231942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.233490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.246811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 398/790] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:27.252626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.333844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.341131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193725f-101d-4a25-985f-e9360b3eb080 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.363325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbdfce-81c0-4289-a1c7-c574c402160b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.379595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.380558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.380756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.381099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.392352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:27.397865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:27.398980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:27.400024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:27.401107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 393/791] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:27.401981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.407228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.408150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.408619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.409055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.418464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.420348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:27.436654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:27.437838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:27.439024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:27.447495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 399/792] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:27.447495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.485573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.522566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8669d-3e29-4c6d-872e-e327ea6c7845 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.562558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.563186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.563462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.563731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.564363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.566016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:27.575423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:27.576466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 02:06:27.577719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:27.584222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 394/793] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:27.584222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.591904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb41b15-863f-4bbf-9fa1-c3f19693e50f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.608528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.634574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.635324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.635669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.635783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.636401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:27.636735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.638209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.647203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 400/794] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:27.647835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.740318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ab70-09ad-424d-ba28-44f6c32b9f77 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.769145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.774821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.775429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.775707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.775974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.776823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:27.782342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.782342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6e80-d22b-47f9-9b6b-737223244fb3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.782676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806974d-1e07-40cc-aa99-6b51d54c9cb6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 395/795] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc3b4-948c-4e83-8e82-4414ce8c2049 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc3b4-948c-4e83-8e82-4414ce8c2049 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc3b4-948c-4e83-8e82-4414ce8c2049 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 396/796] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:27.802334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc3b4-948c-4e83-8e82-4414ce8c2049 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.827765 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.828431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.828748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.829022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.829710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:27.830023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.831391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.840939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 401/797] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:27.840939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:27.888938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:27.913143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc3b4-948c-4e83-8e82-4414ce8c2049 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.950126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.950756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.951037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.951135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca27422-1a0e-46be-a167-a0d9ce16324c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:27.951334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.962822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:27.964275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-60ff448e-0f2e-4675-b621-02a8e6c4eee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:27.979345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:27.980349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:27.981264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:27.981264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:27.981264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:27.981264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 397/798] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:27.981905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.982199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.982463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:27.995941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:27.996574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:27.996883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:27.997159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.000138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:28.001345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:28.009244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.010976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:28.027182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:28.028319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:28.029492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:28.029492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:28.029492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:28.029492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 402/799] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:28.029492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.039618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.074240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 398/800] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:28.075075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.132644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61062e-d359-4250-9f62-9e30671f6175 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.141585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.167860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.168457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.168767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.169058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.169662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:28.169960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.171348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.176018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 403/801] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:28.176805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.184521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4320e-7480-46d0-971c-ac2ded013fcb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.218714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.219326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.219611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.219896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.220543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:28.220881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.222252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb82cf6-6570-44e0-a4a9-d44a89f27f92 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.233268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 399/802] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:28.234423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7304-4655-422a-a940-b897b3007a34 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.234699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7304-4655-422a-a940-b897b3007a34 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.234970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7304-4655-422a-a940-b897b3007a34 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.236290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 400/803] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:28.236833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be7304-4655-422a-a940-b897b3007a34 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.252061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.283895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e912d7-749f-46a4-9a18-47d3f3cbbac4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.318108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.319262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.319887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.319887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.320405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:28.320738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.322097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.333440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 404/804] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:28.334205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.339043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be7304-4655-422a-a940-b897b3007a34 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.356153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.372690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.373319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.373596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.373866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.385373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:28.390361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:28.391454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 401/805] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:28.392322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.439153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3785875-341c-4289-b99b-fb64e81a97bd None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.467137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.472876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.473485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.473852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.474094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.474758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.476811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.495842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eb497-d19f-46d0-8d5d-c3dbaa988bcf None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.528718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.529340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.529617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.529885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.539599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.544610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5d1ab5b3-1dc8-4083-b4a9-6ff1b1126da8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:28.552100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:28.553244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:28.554787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 402/806] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:28.555570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.555570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.555720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.572948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:28.574148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:28.601570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 405/807] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:28.602779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.611004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.658590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 403/808] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:28.660035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.727792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7309b-97fa-48cd-b355-156e66607485 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.770721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.771361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.771646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.772442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.780947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.782261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.783260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1585e5-5b0c-40d3-ba20-427fca42581d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:28.794912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:28.796037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:28.797134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:28.798874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 406/809] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:28.799800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.823342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.823958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.824237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.824507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.825209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:28.825511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.826865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.835681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 404/810] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:28.836301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.856953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.865361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:28.911595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e00d5-d06a-4ab1-96e2-331f3fe209ed None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.946014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dfe9a0-d70e-47f9-8bbb-967440f77d19 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:28.950581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.951197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.951478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.951753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.952491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:28.952864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.954473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.973476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 407/811] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:28.974305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:28.985775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:28.986707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:28.986707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:28.986949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] PATH_INFO: `/v3/services/network` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:28.987533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:28.989055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:28.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:29.004339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:29.005448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 405/812] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:29.006982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.098030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2d264-ffd8-4261-930c-77192cf7b0d5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.099364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.101716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.110548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff8ed-6c6f-43fa-832d-2db9cfcd2760 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.136456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.137187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.137528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.137859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.138616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:29.139078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.140556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.148052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.148704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.149000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.149269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.152816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 408/813] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:29.153871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.167635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.169079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:29.183803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:29.185001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:29.186056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:29.186056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:29.186056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:29.186056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 406/814] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:29.186056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.266509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26f19e-9d4d-4e4f-af5e-dd731736f060 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.292908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a94e0d-4e31-4bc0-a1c7-fe9d292bc09e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.303481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.304607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.305091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.305620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.329927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.330778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.331236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.331684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.332570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] RBAC: Adding query filter params (name=network) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:29.333094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.335136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.339998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.344763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 407/815] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:29.346047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.356071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.381260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.383027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.448227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d302-c713-4e28-b65a-6c2ba4b37545 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.482716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 409/816] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:29.483713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.488520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.489216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.489515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.489851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/171d2d63e59b41a783cdbc61db788122/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.492532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=171d2d63e59b41a783cdbc61db788122, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.494004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.546401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.587242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 408/817] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:29 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/171d2d63e59b41a783cdbc61db788122/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:06:29.588070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.589387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7efcc-7bef-418c-af9a-b79f28af7723 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.638799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.639421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.639696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.639967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.640625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:29.640954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.642340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.660812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 410/818] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:29.660812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.681127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.723668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ff996-62f6-4215-943e-263dfb049f7b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.774274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.775056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.775459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.775742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.776397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:29.777399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.778116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.789683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 409/819] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:29.790985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.811608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6d49-511f-4d6e-9753-b56442342926 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.847079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.862997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.863897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.864173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.864703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.879370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.880875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:29.902849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:29.904043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:29.905186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:29.905186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:29.905186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:29.905186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 411/820] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:29.905457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:29.917600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32459d72-1c74-4575-95dd-2ba95d848bc4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:29.958386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:29.967727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:29.968556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:29.969235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:29.972561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:29.982198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:29.983901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:30.000197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:30.001919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 410/821] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:30.003229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.030858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffe408-4bc4-4133-bb08-075e67220a71 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.068412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.069545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.070059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.070563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.071467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.073143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.126797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d991f-fa5f-4fec-bd76-ebb095a5e65c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.155324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 412/822] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:29 2022] POST /identity/v3/endpoints => generated 337 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:30.156695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.188457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.189116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.189389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.189735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.190376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:30.190680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.192156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.204804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:30.226032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 411/823] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:30.231625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.298158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d314e-68e0-4a79-abe5-158505d92e7a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.333083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:30.342700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.343501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.343897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.344265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.345531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:30.345899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.347458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.372395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 413/824] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:30.378658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.474411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161c8c7-8415-413d-a559-2d3d7a59d05a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.489098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:30.501575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445ed3e-2871-4193-b154-536419a927aa None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.519959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.520972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.521449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.522558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.530784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.532591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.546169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.546801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.547077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.547344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:30.547794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:30.548854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:30.549765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:30.552072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 412/825] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:30.558661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.559877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.561360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:30.586801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:30.587804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:30.588749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:30.588749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:30.588749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:30.588749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 414/826] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:30.597827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.692405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419a37-a2fd-41e2-9cc4-7545149a2c8c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.705188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:30.720070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc302-c26c-4a31-800e-e3f3955e4f67 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.742361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.742981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.743263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.743995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.744728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:30.745194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.746638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cbfcf-1224-4812-89b9-0e9d49095ef2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.759070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.759678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.759952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.760176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 413/827] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:30.760281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.765497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:30.765497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.765732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3e56-8f67-4913-9e82-1897b50a2f77 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.765732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3e56-8f67-4913-9e82-1897b50a2f77 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.765732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3e56-8f67-4913-9e82-1897b50a2f77 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.765955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd72a5-fbe8-436e-8d35-d7afbb9ffe31 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.767171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 414/828] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:30.767929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3e56-8f67-4913-9e82-1897b50a2f77 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.783897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 415/829] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:30.783897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.783897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.783897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.789868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:30.804435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:30.804435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:30.875470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 416/830] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:30.876836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.884056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3e56-8f67-4913-9e82-1897b50a2f77 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:30.921327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:30.921928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:30.922204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:30.922473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:30.923189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:30.923491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:30.924928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:30.940698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 415/831] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:30.940698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:30.970631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:31.029101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae6f7-2cb5-4c40-822a-91c409da438f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.057562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a588c7c-aed7-46d6-9c2a-6862b7f370ae None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.078534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:31.086770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.090447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.090447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.090447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.097929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.097929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.098203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.098281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.098281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.100367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.100477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.100569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:31.111399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:31.112410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:31.113290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:31.114051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:31.114867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:31.116411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:31.118059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 416/832] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:31.118212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 417/833] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:31.120330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:31.125250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:31.246372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84a24-559a-42b0-a4bd-fa271f283c57 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.246570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:31.254964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3451e8-51e5-44b0-b51a-df43b36dc368 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.289631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.290261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.290534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.290811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.291478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:31.291996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.293169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.298397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 418/834] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:31.304277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:31.305270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.306147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.306624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.307397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.309339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:31.309867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.311498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.328641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 417/835] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:31.336422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:31.368794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:31.420953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178728-a10a-48b3-9295-47998fb42daa None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.457501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.458118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.458391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.458656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.459336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:31.459642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.461268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299784f-2869-499e-9663-5961184b5a26 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.472704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 419/836] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:31.472911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5163a5-e64d-46fa-82ad-402ff42dd2b8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:31.510492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:31.511252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.516694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.516694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/76ca42843b2941e0860bca74f5d7c5f6/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.529321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=76ca42843b2941e0860bca74f5d7c5f6, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:31.530852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb5ada-0ccb-444e-a972-19f04f8d9ddd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:31.605636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 418/837] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:31 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/76ca42843b2941e0860bca74f5d7c5f6/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:06:31.769896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc1c6-0f20-4db1-b703-5f95e65a6508 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.770237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc1c6-0f20-4db1-b703-5f95e65a6508 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.770434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc1c6-0f20-4db1-b703-5f95e65a6508 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.772767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 420/838] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:31.778389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:31.778935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:31.779405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:31.787003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:31.801886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:31.803142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:31.877678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 419/839] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:31.901768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afc1c6-0f20-4db1-b703-5f95e65a6508 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:32.033899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afc1c6-0f20-4db1-b703-5f95e65a6508 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:32.062774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:32.072480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:32.072480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:32.072480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:32.072480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:32.083031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:32.088850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:32.089774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:32.090611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 421/840] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:32.101316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:32.207730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce3cc1-48a6-48d9-801f-06f3bf05af1b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:32.251439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:32.252363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:32.252917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:32.253382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:32.254288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:32.254834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:32.257043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:32.266328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 420/841] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:32.270653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:32.273468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:32.398476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c6834-1c78-4869-89b7-eb58f0ba83a1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:32.464126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:32.465985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:32.466444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:32.466904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:32.486127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:32.487765 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:32.512018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:32.513076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:32.513939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:32.513939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:32.513939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:32.516145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 422/842] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:32.523643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:32.685376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8a3c6-0ade-4edc-9654-58c634384732 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:32.749860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:32.752144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:32.752915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:32.753680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:32.754816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:32.756633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:32.760592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:32.799796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 421/843] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:32.800941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:32.884961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:32.963518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366297-dcd9-4437-bbf5-926652b37e01 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:32.999419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.000101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.000349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.000635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.008552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.010016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:33.021521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:33.023090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:33.024030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 423/844] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:33.030742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.092881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.141266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3be0a-07e3-44a3-ac8b-eac79319f59e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.179576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.180546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.181048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.181530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.182471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:33.182972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.184582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdc24a-0d67-4cc0-8eda-dfa5ad2cc820 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.203877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 422/845] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:33.210933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.242867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63086e-1ae8-4e71-9df8-e2398ec6286d None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.243479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63086e-1ae8-4e71-9df8-e2398ec6286d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.243948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63086e-1ae8-4e71-9df8-e2398ec6286d None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.245768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 423/846] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:33.253299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.253537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.253799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.277727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:33.279325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:33.322485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.331406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0baa2-f7ef-4f8a-bd44-79f36f6e9441 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.367783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.368377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.368775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.369118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.374478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 424/847] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:33.396533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.452883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.455519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.456292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.514025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df89b3-40a0-4afb-b388-ed59383ebfc7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.559071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.559737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.560133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.560484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.561181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.562688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:33.572774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:33.573790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:33.574668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:33.577352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.579317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 425/848] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:33.580636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.602311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 424/849] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:33.607932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.706238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209530b-9e4e-4c93-abbf-d6a8c75a14a4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.716502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.727460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae12c21-c72b-426a-af98-2e191afae5c5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.742463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.743066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.743343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.743613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.744240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:33.744540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.746072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a54c8-8177-4426-b7db-29a1095dc8f8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.751189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 426/850] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:33.752417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd759d-b419-49a6-a4da-8073a6bf2aee None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.752855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd759d-b419-49a6-a4da-8073a6bf2aee None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.753140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd759d-b419-49a6-a4da-8073a6bf2aee None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.754938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 427/851] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:33.756178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d115a-3e0f-40e8-a2a6-a3b5fdacdddd None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.756460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d115a-3e0f-40e8-a2a6-a3b5fdacdddd None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.756760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d115a-3e0f-40e8-a2a6-a3b5fdacdddd None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.757959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 428/852] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:33.758718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d115a-3e0f-40e8-a2a6-a3b5fdacdddd None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.766549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.767114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.767391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.767668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.778327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.779766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-aed01c08-beb4-4845-8091-b26ce2c575a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:33.794976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:33.796110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:33.797253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:33.797253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:33.797253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:33.800763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 425/853] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:33.801872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca38-7b7d-4906-a41c-1e4610be4af7 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.802153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca38-7b7d-4906-a41c-1e4610be4af7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.802425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca38-7b7d-4906-a41c-1e4610be4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.820370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bda1ca38-7b7d-4906-a41c-1e4610be4af7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:33.821587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bda1ca38-7b7d-4906-a41c-1e4610be4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:33.861297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d115a-3e0f-40e8-a2a6-a3b5fdacdddd None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:33.862801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.888096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 426/854] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:33.889408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.889687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.889956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.903861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:33.904464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:33.904772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:33.905044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:33.905669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:33.905967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:33.907334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:33.908297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:33.909569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:33.911824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 429/855] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:33.913159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:33.951975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:33.990536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 427/856] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:33.991576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.031353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c23f-394a-469f-b7b5-b2f2bc622792 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.079738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.080359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.081109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.081389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.082049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:34.082354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.084635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.096611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 430/857] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:34.097816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.113783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a403d-bd22-4193-93b7-e0ae3377722c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.117965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.153152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.153760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.154033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.154304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.154943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.156294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.196342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.202043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0e1d2-ab23-4b3e-8f93-49a0455bba86 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.235023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.235648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.235925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.236196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.246944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:34.256972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 428/858] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:33 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:34.256972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:34.257462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:34.258544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:34.259580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 431/859] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:34.264759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.396113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3362445-3c3f-4258-9a43-f42fb63e6c3b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.404864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b45f70-1806-4706-9ce7-20f894417120 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.437074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.438081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.438507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.438973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.449669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.450224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.450845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.451126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.451397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.452065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:34.452368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.452732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.453766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.464448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 432/860] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:34.465813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:34.467021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:34.467927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 429/861] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:34.474597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.475251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.497037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.521059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.595712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6222f-5c60-4ebc-9e39-7962a7dd3c35 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.602299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1734a-38d5-48a9-8e59-6f933c2844c0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.639135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.639752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.640039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.640315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.641055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:34.641439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.642801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.643268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.644080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.644557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.644997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.662932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.663109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 433/862] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:34.663222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.665314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:34.684024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:34.685086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:34.686006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:34.686006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:34.686006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:34.688473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 430/863] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:34.694330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:34.756426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.780134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401cef4-77a1-4472-8446-d3b25a6394e8 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.786682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:34.818219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.819817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.820161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.820504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.821322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=171d2d63e59b41a783cdbc61db788122) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:34.821696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.823196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.827860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc5ad-ee99-4fe7-a3c4-f6c508f8c8e7 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:34.837112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 434/864] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/role_assignments?user.id=171d2d63e59b41a783cdbc61db788122&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:34.867483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:34.868438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:34.868988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:34.869444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:34.870350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:34.870846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:34.872424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:34.887409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 431/865] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:34.893566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.009311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643483-b0ad-426d-a1db-eb1ae07ac70e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:35.046919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:35.047956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.048466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.048980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.058834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:35.060745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:35.073985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:35.074974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:35.075905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:35.078432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 435/866] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:35.083747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.191365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d935d8-9fb4-4d47-87a1-ada4eeeb0c81 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:35.225975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:35.226814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:35.227443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.227717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.227985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.228733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:35.229041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:35.230461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:35.249331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 432/867] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:35.255412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.259081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:35.365255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90938e9-24a4-495e-b431-75c08a85aa94 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:35.398221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:35.398830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.399104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.399372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.469119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:35.470595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:35.473947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:35.576415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 436/868] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:35 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:35.581964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.693012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e866ffd-8ec1-4817-babc-05d669dcc481 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:35.729454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:35.730416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.730902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.731398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.742292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:35.743973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:35.759772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:35.760745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:35.761805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:35.761805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:35.761805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:35.763477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 433/869] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:35.769199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.873327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2e82c-4fdd-46ea-87ce-43c7ef50880a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:35.875120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:35.908697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:35.910500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.910967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.911390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.912749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:35.913206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:35.914987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7d25d-2415-447f-864c-a7c8cdffcefb None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:35.927317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 437/870] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:35.932764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:35.981055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:35.987912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475824d4-0d6a-4632-93fa-204dcdf92c15 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.988203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475824d4-0d6a-4632-93fa-204dcdf92c15 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.998079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475824d4-0d6a-4632-93fa-204dcdf92c15 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:35.998079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 438/871] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:35.998079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:35.998079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:35.998079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.016983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:36.018352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:36.050912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a4a25-3fba-4488-8bc4-44ae724c9e5e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.097558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 439/872] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:36.100801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.101706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.102158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.102597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.110134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.111810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.123164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:36.128062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:36.129160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:36.130054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:36.133083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 434/873] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:36.138835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:36.236377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:36.247417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cedbc28-1759-49f4-a240-0d001904abf7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.277475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9a8f-423c-431c-8947-dddb1aa53bf0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.280051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:36.286992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.288477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.289594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.290297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.291566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.293103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac33ad-49d2-4ad9-899c-e008270d3fcf None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.319459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.320070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.320351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.320623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.321342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:36.321646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.323074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.336081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 435/874] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:36.341770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:36.424179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 440/875] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:36 2022] POST /identity/v3/services => generated 269 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:06:36.426798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211ef0-804b-4a36-8aa4-f05507c70f04 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.427625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211ef0-804b-4a36-8aa4-f05507c70f04 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.428227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211ef0-804b-4a36-8aa4-f05507c70f04 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.430583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 441/876] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:36.438483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.438807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.439037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.455890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d5b95-2b17-4266-9634-0487756dd955 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.478773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:36.479993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:36.509045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.510011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.510499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.511103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.512083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=76ca42843b2941e0860bca74f5d7c5f6) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:36.512694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.514381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.526905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 436/877] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:36 2022] GET /identity/v3/role_assignments?user.id=76ca42843b2941e0860bca74f5d7c5f6&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:36.565758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 442/878] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:36.588397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:36.624817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:36.714513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde9357-3e05-453f-b384-136885d4268b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.751118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.751751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.752032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.752302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.752921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.754350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:36.763842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:36.764900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:36.765743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 437/879] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:36.772785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:36.862136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:36.880542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fb967-dadf-488f-bbd8-39eebef74ee3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:36.917507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:36.918130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:36.918434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:36.918712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:36.919411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:36.919716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:36.921207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f06b-57b8-4820-91fe-eae5f7be7578 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:36.926343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 443/880] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:36.932876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.003669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33fa4-b13f-4bdb-bc0a-5569c61e6a21 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.004024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33fa4-b13f-4bdb-bc0a-5569c61e6a21 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.004376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b33fa4-b13f-4bdb-bc0a-5569c61e6a21 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.006620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 444/881] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:37.011603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.011884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.012150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.030101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:37.031204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:37.036942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:37.057308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb24e9-6946-48c5-857f-0ae16b441e64 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.103695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.104374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.104586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.104890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.105526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:37.105835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.107200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.112448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 438/882] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:37.113181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 445/883] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:37.118634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.130251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.229977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:37.243820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea4a0-1119-4794-8883-35d509d6cb18 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.261619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505ca6-91ec-4711-86a0-5ad3276db758 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.286519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.287052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.287864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.288141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.289573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] RBAC: Adding query filter params (interface=public, service_id=e56ea27f90e449ba96ee7240d711ba9c, region_id=RegionOne) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:37.290048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.291706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.300840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 439/884] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:37 2022] GET /identity/v3/endpoints?service_id=e56ea27f90e449ba96ee7240d711ba9c&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:37.305226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.311171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.311809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.312084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.312352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.312991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.314349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.325489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:37.328296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:37.329332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:37.330188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:37.332746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 446/885] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:37.337666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.424156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacbc4-8a65-4abd-9b1e-91f89fa7c634 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.442582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49046c-3b25-4d6e-abd6-82a1b5f69b2b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.464103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.464830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.465123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.465512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.466135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.467648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.479429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.480447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.480975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.481171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 440/886] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:37 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:06:37.481642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.482511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:37.483042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.484700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d392818-16a6-4efd-8732-35ac741a0c17 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.491024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 447/887] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:37.497248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:37.591752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:37.616920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8eab00-f12a-4810-b62f-7e2347124a93 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:37.656340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:37.657912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:37.657912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:37.657912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:37.658717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:37.659030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:37.660546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8c668-6b5b-457b-b082-b241c0d18498 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:37.667077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 441/888] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:38.574413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f4e0-e516-4f39-98d8-437b698b72e3 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:38.575447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f4e0-e516-4f39-98d8-437b698b72e3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:38.575940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825f4e0-e516-4f39-98d8-437b698b72e3 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:38.577782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 448/889] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:38.583180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4054e-54bf-4683-a7c1-319b2ad6f916 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:38.583720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4054e-54bf-4683-a7c1-319b2ad6f916 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:38.584464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4054e-54bf-4683-a7c1-319b2ad6f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:38.603884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d3a4054e-54bf-4683-a7c1-319b2ad6f916 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:38.605820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d3a4054e-54bf-4683-a7c1-319b2ad6f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:38.681571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 442/890] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:38.705564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f4e0-e516-4f39-98d8-437b698b72e3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:38.711733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:38.732823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427259fb-50ab-4f30-915a-b6131c5f46c5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:38.733154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427259fb-50ab-4f30-915a-b6131c5f46c5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:38.733434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427259fb-50ab-4f30-915a-b6131c5f46c5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:38.734908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 443/891] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:38.739956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:38.740304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:38.741089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:38.763510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:38.765054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:38.828512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825f4e0-e516-4f39-98d8-437b698b72e3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:38.860042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 444/892] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:38.869742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:38.870674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:38.871159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:38.871674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:38.872461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:38.874051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:38.879722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:38.882471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:38.886100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:38.887031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:38.887931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:38.890746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 449/893] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Aug 8 02:06:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:38.898200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:38.992393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.002492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77c5b6-c6fc-460f-aa4b-d2d80caf0bd5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.043611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.044246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.044525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.044832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.045193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34816471-27bd-4afc-a933-4c3fef997943 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.057149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:39.062174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:39.063127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:39.064043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:39.068811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 445/894] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:06:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:39.069766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.093061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.094011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.095042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.095541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.096587 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:39.097309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.101283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.112839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 450/895] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:39.114601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.143857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.188416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb159cd0-2320-41cc-a9b4-bc20a92f276e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.227814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.228552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.228877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.229149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.230132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:39.230380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.231800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.246405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 446/896] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:39.247215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.253230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b236505-a8d0-4086-9871-a86da0d34897 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.297481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.305593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.306774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.307299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.308023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.309309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:39.310095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.312373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.324944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 451/897] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:39.326674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.374920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95113169-2ae3-4371-8441-dac4d4c6cc7e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.406634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.421378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.422579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.423013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.423434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.438053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.438852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0b97-cbe9-4c5e-bfaf-d0a04a18da68 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.439894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:39.460138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:39.461346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:39.462457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:39.462457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:39.462457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:39.464694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 447/898] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:39.470908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.483734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.484593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.485384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.485817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.487100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] RBAC: Adding query filter params (interface=public, service_id=035604f997a441baa42f3355d3fa3f88, region_id=RegionOne) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:39.487563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.489696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.503025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 452/899] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/endpoints?service_id=035604f997a441baa42f3355d3fa3f88&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:39.504057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.570896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.601148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463d8a-587e-47c6-8863-0c8a0b65631c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.618481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139236ba-c99b-43b0-82f7-92c0917dbe91 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.637636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.638205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.638638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.638746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.639352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:39.639655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.641063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee336753-4aaf-4b73-b542-e20802312b35 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.658862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 448/900] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:39.658862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d0d4-5a2a-4757-bc2c-fa1a95c43041 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.658862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d0d4-5a2a-4757-bc2c-fa1a95c43041 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.658862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d0d4-5a2a-4757-bc2c-fa1a95c43041 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.658862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 449/901] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:39.665361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144d0d4-5a2a-4757-bc2c-fa1a95c43041 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.665441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.665441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.665441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.665441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.665713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.665713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09474-87b6-453a-89ab-5ab0213e9c37 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.680891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 453/902] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:06:39.682342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4a95-1c38-4411-b59e-7200f0e3637f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.682632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4a95-1c38-4411-b59e-7200f0e3637f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.682913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4a95-1c38-4411-b59e-7200f0e3637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.708915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.716535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-175f4a95-1c38-4411-b59e-7200f0e3637f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:39.717971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-175f4a95-1c38-4411-b59e-7200f0e3637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:39.787200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144d0d4-5a2a-4757-bc2c-fa1a95c43041 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:39.800746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 454/903] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:39.800746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a86d98-0118-4708-97c8-ee1f6bf1574c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.800746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a86d98-0118-4708-97c8-ee1f6bf1574c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.800746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a86d98-0118-4708-97c8-ee1f6bf1574c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.800746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 455/904] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:39.802374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.803215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.803695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.824427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:39.825679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:39.837054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:39.837696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:39.837976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:39.838248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:39.846481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:39.847929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:39.864059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:39.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:39.866676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 450/905] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:39.867265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.886603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.912842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 456/906] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:39.912842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:39.984770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:39.991990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f7a20-1d9a-46be-a613-974de445d35c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.015778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128697b2-d506-4bf1-841d-ed03b298ace0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.035014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.036070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.036356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.036633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.037727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.039884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.049375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.049982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.050257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:40.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:40.051427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 02:06:40.052278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:40.054778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.054778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:40.054778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.054965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 451/907] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:40.054965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.058228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.082106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 457/908] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:40.082106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.124342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:40.170207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4e482-b021-45fc-ad04-e7b675dbc95c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.183017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271315-a188-4b3f-af12-f95700ca0923 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.206932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.207793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.208255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.208531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.209252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.210612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.217800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.218413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.218687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.218956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.219589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:40.219892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.221266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.226822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 458/909] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:40.227589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.273076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 452/910] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:40 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:40.273953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.300292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:40.374319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45589-392f-4bad-872f-aab2e3a5acdd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.386302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:40.407666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51195b-140e-4270-abf0-fad0572cd755 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.424621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.425927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.426443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.426916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.450175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.451486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.451486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.451486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.452085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:40.452398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.453854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.464486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 453/911] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Aug 8 02:06:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:40.465429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.514292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.515770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.566154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:40.663112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509798f-52fa-422f-9022-78ada0ecbf90 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.663112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.663112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.663112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.664299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.664299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.664299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.664507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 459/912] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:40 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:40.679094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:40.688589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:40.760863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 454/913] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:40.813648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ff30f-5f87-4bce-ba52-6501190ad18c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:40.871987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:40.872973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:40.873452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:40.873880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:40.887697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:40.889743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:40.911281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:40.912700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:40.914275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:40.914275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:40.914275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:40.916284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 460/914] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:40.922223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:41.052894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699259b8-903b-495a-807e-6eed69ab28f5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:41.092333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:41.092985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.093261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.093533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.094307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:41.094612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:41.095978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:41.098505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:41.114555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 455/915] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:41.121131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:41.192451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:41.241388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45a9cc-0aa5-4c46-a133-c6f0ddb14790 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:41.278635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:41.279274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.279754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.280082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.287308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:41.289063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:41.300792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:41.301756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:41.303710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 461/916] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:41.309141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:41.433904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174c85a-820c-42f9-9445-8bd97a6c2d8e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:41.481196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:41.481819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.482099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.482419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.483131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:41.483442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:41.484918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63676bb1-7d0f-4b6f-9825-02dcadf527c0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:41.499819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 456/917] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:41.505624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:41.616488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d01ea-c4e0-40ed-a6ba-aa343f85e813 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:41.635181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:41.654307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] RBAC: Adding query filter params (role.id=875680c7c12b46e088ceeb8b0be7d817, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=76ca42843b2941e0860bca74f5d7c5f6) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:41.663048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b97bd-b4d7-41b7-ab69-c6282dbb5234 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:41.666012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 462/918] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:41 2022] GET /identity/v3/role_assignments?user.id=76ca42843b2941e0860bca74f5d7c5f6&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=875680c7c12b46e088ceeb8b0be7d817 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:41.825242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710b8f-571b-401c-8cb9-ea1f7f1d2c15 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.826237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710b8f-571b-401c-8cb9-ea1f7f1d2c15 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.826755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710b8f-571b-401c-8cb9-ea1f7f1d2c15 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.828556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 457/919] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:41.834169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:41.834571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:41.834945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:41.854841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:41.855949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:41.928881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 463/920] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:41.952133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710b8f-571b-401c-8cb9-ea1f7f1d2c15 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.015928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:42.065148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710b8f-571b-401c-8cb9-ea1f7f1d2c15 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.105826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.106384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.106666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.106943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.107537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.108938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:42.118162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:42.119160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.120223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 458/921] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Aug 8 02:06:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:42.125302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.232397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5fa32-cc04-42a9-87b6-81e50ecf81e9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.242598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:42.268788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.269397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.269679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.269950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.270699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:42.271475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.273420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.278796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 464/922] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:42.284507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.394940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c566db-3b92-4c3c-8e53-5e0fbd416158 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.416526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:42.432611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.433581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.434099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.434560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.435505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:42.435987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.437640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a92b77-38e5-415c-9ebe-2820902e0874 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.443888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 459/923] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:42.449916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.455485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c51e4-5686-4305-a5bb-6d005b8925bc None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.455776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c51e4-5686-4305-a5bb-6d005b8925bc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.456057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c51e4-5686-4305-a5bb-6d005b8925bc None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.457656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 460/924] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:42.462656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.462946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.463225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.485285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:42.486500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:42.566567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b9378-a31f-46dd-bcaa-c62b7d2da8ce None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.584486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 461/925] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:42.608835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.616095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.616884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.617265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.617674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.618625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.621136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.721772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 465/926] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:42.725611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabef9b1-30a4-4ead-9430-3d62ae46be12 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.744552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:42.776543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.777598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.778044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.778481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.779260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.780967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:42.792105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:42.793471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:42.794424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:42.796993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 462/927] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:42.804855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:42.838470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:42.921662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06cb8-f247-4694-9688-94bae1c304e0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:42.958222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:42.959372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:42.959658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:42.959924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:42.960629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:42.960964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:42.962339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b55c8-3f1d-47ed-8002-5ca465f0b380 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:42.967758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 466/928] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:42.973586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.057030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e46a3-11d6-4aef-8c31-1966d39d336b None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.057397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e46a3-11d6-4aef-8c31-1966d39d336b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.057601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e46a3-11d6-4aef-8c31-1966d39d336b None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.059065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 467/929] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:43.068684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.068684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.068684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.089717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:43.091899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:43.093110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5ceac-7a46-46bd-b75b-53acc0323cf2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.135025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.136065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.136550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.137046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.137889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:43.138411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.139992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.145777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 463/930] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:43.152556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.184748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 468/931] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:43.207912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:43.211251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.237919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:43.283581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc62-47c7-4355-9f5d-f2a7c1043bda None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.324402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.325525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.326030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.326528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.327452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] RBAC: Adding query filter params (interface=public, service_id=4d7fa75eb78842339cbcdc072ce8419d, region_id=RegionOne) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:43.327966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.328113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7826a0-38d3-44d7-ba4c-8deea466129c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.329940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.340970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 464/932] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:43 2022] GET /identity/v3/endpoints?service_id=4d7fa75eb78842339cbcdc072ce8419d&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:43.342185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.372045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.372734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.373022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.373346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.374265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.376402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.379999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING keystone.server.flask.common [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 02:06:43.432236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.sql.core [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:06:43.432236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:06:43.432236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [parameters: {'id': '1dd3d05310a94cc29fbb7fd4aafffa52', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:06:43.432236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80538) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1dd3d05310a94cc29fbb7fd4aafffa52.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1dd3d05310a94cc29fbb7fd4aafffa52. Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:06:43.449352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application result.read() Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 02:06:43.450561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application yield resource Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise self.value Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application session.commit() Aug 08 02:06:43.451539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:06:43.452492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:06:43.453455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application result.read() Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:06:43.456205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [parameters: {'id': '1dd3d05310a94cc29fbb7fd4aafffa52', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:43.458378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:43.461394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1dd3d05310a94cc29fbb7fd4aafffa52. Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 469/933] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Aug 8 02:06:43 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:43.464921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.475047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b5aa9-3216-4333-ab7b-d330db61e579 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.477595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:43.522320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.523892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.524539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.525345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.526480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.528213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.536707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 465/934] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:43 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:06:43.553372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:43.586800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe7070-269b-445e-a226-f6fdb252a795 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.636412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.637368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.637884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.638256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.647199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.649239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-3f0005d2-e71e-40db-abfa-a7a534f87a45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:43.663522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:43.665440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:43.666543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:43.667871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 470/935] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:43.673670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.727542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2513-333f-4b22-85c3-63152d573054 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.728063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2513-333f-4b22-85c3-63152d573054 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.728307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2513-333f-4b22-85c3-63152d573054 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.733242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 471/936] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:43.738884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.739311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.739899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.761338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:43.763003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:43.799597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad26ed-1185-4b7a-a868-4090d6b4ce63 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:43.833448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 472/937] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:43.850478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:43.851301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:43.851578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:43.851848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:43.852567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:43.852907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:43.855105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:43.856944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:43.878044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 466/938] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:43.937716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:43.979664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dea0f-4318-4bbd-a68b-256c68d9a0ba None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.018336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.018987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.019264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.019535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.032283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:44.037202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:44.038147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:44.039281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 473/939] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:06:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:44.044615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.077706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:44.154468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faafbc-6618-4739-92d9-bbfcec1321b6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.190413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.191077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.191358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.191632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.192348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:44.192654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:44.194425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:44.206000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 467/940] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:44.211622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.317285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4528d99-08ca-44ae-818e-742b3540d16d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.345406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:44.354092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.354733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.355014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.355291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.366070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:44.367532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:44.382880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:44.383851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:44.385169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:44.385169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:44.385169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:44.385169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 474/941] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:44.391341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.495568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:44.508902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d0842-efb7-4e11-aa75-dbb682bd35a3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.545832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.546858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.547377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.547849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.548725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:44.549241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:44.550801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:44.569207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 468/942] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:44.571364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.687878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5d1e7-0368-469f-a4f8-819a04e5f4a9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.727402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.728368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.729301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.729734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.738849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:44.740599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-db9bffc7-88d4-4f32-9339-fb59f3ac5d2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:44.752835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:44.754038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:44.755225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:44.758299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 475/943] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:44.767091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.868467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc00e-b0f5-4fc1-b6f9-a5d2a331f2ac None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:44.903467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:44.904080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.904357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.904630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.905442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:44.905757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:44.907182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:44.916399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e31e-a521-4701-ae1b-ddb782f13484 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.917083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e31e-a521-4701-ae1b-ddb782f13484 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.917589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9e31e-a521-4701-ae1b-ddb782f13484 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.919379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 476/944] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:44.924480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:44.925019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:44.925499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:44.928363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 469/945] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:44.931284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:44.940875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:44.950933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:44.952880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:44.997265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.042047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 477/946] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:45.054794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c41612-e1d2-49c7-a0f5-9c45e4fc94e6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.065425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.093996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.095072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.095660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.096177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.108253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.110145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:45.128416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:45.129460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:45.130336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:45.130336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:45.130336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:45.138646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 470/947] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:45.140422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.178598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1278c9d-847c-4234-98fa-36acf977e7b6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.207028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.217800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.218374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.220318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.220318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.220318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.221009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:45.230081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:45.231096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:45.231915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.236530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 478/948] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:45.238354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.262341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb38f7c-4054-403b-82ed-258d7fbc5899 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.277058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.301413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.302972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.303557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.304129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.305200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:45.305717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.307543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.325742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 471/949] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:45.327397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.350330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1b47f-fbd5-4961-a108-d882530dd5e2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.389269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.389743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.390024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.390295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.390930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:45.391262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.392591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.403271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 479/950] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:45.404213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.438881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba9129-d9d7-4327-ae6c-af17449a025e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.460433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.479234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.480306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.480837 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.481412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.488996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.490850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:45.503702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:45.504799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:45.506340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:45.514566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 472/951] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:45.516049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.516642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.519778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff2173-3f53-4831-85bf-2e7511febcb3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.560635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.561355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.561639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.561912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.562546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:45.562855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.564324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a2c56-9246-471b-853c-1095ef656910 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.569941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 480/952] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:45.627732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60446-5ab7-42fd-a6df-c980c50ab870 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.628109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60446-5ab7-42fd-a6df-c980c50ab870 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.628321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60446-5ab7-42fd-a6df-c980c50ab870 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.629929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 481/953] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:45.635592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.636096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.636376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.642634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b222f-2885-4a47-a645-1b9361d2f464 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.656801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:45.664307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:45.684586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.685629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.686227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.686712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.687642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:45.688220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.689969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.705454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 473/954] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:45.711554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.752759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 482/955] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:45.770451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.772030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:45.851687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d702de-ddce-4d94-9423-e05eead57a1c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.892094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:45.895625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.896604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.896863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.897084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/76ca42843b2941e0860bca74f5d7c5f6/roles/875680c7c12b46e088ceeb8b0be7d817` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.899281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=76ca42843b2941e0860bca74f5d7c5f6, role_id=875680c7c12b46e088ceeb8b0be7d817)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.900884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc176-4714-46f1-9db0-fe7cb5e36e1c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.903768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65f1a-b8b4-44eb-b875-d816c7105d4a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:45.939592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:45.940372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:45.940789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:45.941161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:45.941848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:45.943491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:45.952943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:45.954284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:45.955410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:45.957288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 483/956] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:45.963583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.017167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 474/957] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:06:45 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/76ca42843b2941e0860bca74f5d7c5f6/roles/875680c7c12b46e088ceeb8b0be7d817 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:06:46.049483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4e78e-e738-4ecd-b91b-85a4865cb9d6 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.049815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4e78e-e738-4ecd-b91b-85a4865cb9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.050033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4e78e-e738-4ecd-b91b-85a4865cb9d6 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.051651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 475/958] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:46.057221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.057860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.057860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.078237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:46.079654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:46.082722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c85ed0-8204-4bdc-835c-683b068ce658 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.123601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.128313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.128593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.128940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.129611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:46.129915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.131894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.137704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 484/959] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:46.143525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.170609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 476/960] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:46.182583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:46.189795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.225340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:46.286153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91850-4dc4-4889-aeb4-20f89b3c35e7 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.318290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29049561-6584-43c0-ac9b-d89467b3f0aa None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.323038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.323660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.323942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.324219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.324902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:46.325209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.326593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.333796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 485/961] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:46.339336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.359953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.360574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.360944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.361221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.361905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.363351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.366612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING keystone.server.flask.common [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 02:06:46.426119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.sql.core [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:06:46.426119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:06:46.426119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [parameters: {'user_id': '04f5f0e8dbbb4e7a98fb6fd9c561e492', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:06:46.426119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 02:06:46.429496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result.read() Aug 08 02:06:46.436545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:06:46.437650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application yield resource Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application next(self.gen) Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise self.value Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application session.commit() Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 02:06:46.438853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.session.flush() Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application compat.raise_( Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 02:06:46.439958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 02:06:46.441795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application conn.query(q) Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application result.read() Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 02:06:46.444314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [parameters: {'user_id': '04f5f0e8dbbb4e7a98fb6fd9c561e492', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:46.447052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 02:06:46.449036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 477/962] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Aug 8 02:06:46 2022] POST /identity/v3/users => generated 160 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:46.451489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.457368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab52366a-6609-4b2d-99db-48244ad79a4d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.515573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.516601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.517015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.517829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.518922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.521588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.550580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:46.560183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c434eb2-8da4-42b0-9c4f-feca3009102c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.604904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 486/963] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:06:46.621938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.623004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.623497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.623950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.633347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.635274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-9fa2c6e5-dfeb-4620-a4b2-b56cd457fb25 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:46.649709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:46.650773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 02:06:46.651773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:46.654615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 478/964] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:06:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:46.661923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:46.780374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ca8e9-55a7-4739-9b0b-5717061dad94 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:46.816717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:46.817640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:46.818265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:46.818739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:46.819634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:46.820286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:46.821937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa14bad-4933-4775-9b98-73c2529d4d35 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:46.847104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:46.860878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 487/965] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Aug 8 02:06:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:47.670800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1ec3-c61a-4f0e-bccc-cdca1d811f6b None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:47.671339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1ec3-c61a-4f0e-bccc-cdca1d811f6b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:47.671339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c1ec3-c61a-4f0e-bccc-cdca1d811f6b None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:47.672747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 479/966] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:47.677968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339da30-7fb4-4306-bdf7-250b482b98d8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:47.678253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339da30-7fb4-4306-bdf7-250b482b98d8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:47.678531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339da30-7fb4-4306-bdf7-250b482b98d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:47.697123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9339da30-7fb4-4306-bdf7-250b482b98d8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:47.698347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9339da30-7fb4-4306-bdf7-250b482b98d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:47.769997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 488/967] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:47.793666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c1ec3-c61a-4f0e-bccc-cdca1d811f6b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:47.901316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c1ec3-c61a-4f0e-bccc-cdca1d811f6b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:47.936159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:47.936802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:47.937086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:47.937362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:47.938023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:47.939446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6d636-b851-4bf4-b305-173ba35ca8f1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:48.020699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 480/968] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:47 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:48.150160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac179212-7154-4a4e-9738-d6b66cdb2e21 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.150711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac179212-7154-4a4e-9738-d6b66cdb2e21 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.150711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac179212-7154-4a4e-9738-d6b66cdb2e21 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.152312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 489/969] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:48.157664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.157963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.158224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.173268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:48.179420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:48.180959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:48.257167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 481/970] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:48.280660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac179212-7154-4a4e-9738-d6b66cdb2e21 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.383402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac179212-7154-4a4e-9738-d6b66cdb2e21 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:48.409980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:48.418409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:48.419041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.422062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.422062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.430626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:48.435641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:48.436549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:48.437409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:48.439331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 490/971] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:48.446125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.547996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65348d07-1aed-41bf-9b79-72c4a3c9155c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:48.578659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:48.589115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:48.593007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18366c-62d7-485f-9cda-26f8a69342ac None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:48.602093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 482/972] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:48.607555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.703151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8644-af28-468d-a1cc-2f69dd64f4f3 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.703646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8644-af28-468d-a1cc-2f69dd64f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.703646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8644-af28-468d-a1cc-2f69dd64f4f3 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.705119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 483/973] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:48.710755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.711066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.711656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.714080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121be1e2-ff86-4c79-b0d2-a56272324a12 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:48.729993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:48.731186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:48.754757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:48.755472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.755791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.756270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.767342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:48.768922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:48.784412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:48.785425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:48.786322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:48.786322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:48.786322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:48.786322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 491/974] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:48.791963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.809900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 484/975] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:48.832257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.837777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:48.902414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55994c9-1cd5-47c3-af8c-918ac56e336e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:48.937690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:48.938289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.938509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ba0b9-675a-4cc8-afd8-9c81c9550162 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:48.938633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.938818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.939452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:48.939754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:48.941147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:48.953145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 492/976] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:48.959376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:48.969188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:48.980384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:48.981046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:48.982753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:48.982753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:48.982753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:48.983578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:48.993680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:48.994680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:48.997967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 485/977] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:49.000978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.066210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.070261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7aba9-1780-4fdc-80e5-e75a1ee48238 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.108242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ef445-e120-494c-9b8d-9fd2f587bc99 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.112645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.113473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.113898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.114339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.125167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.126878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:49.140737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:49.141694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:49.142618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 493/978] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:06:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:49.145070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.145694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.146534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.147009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.147762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:49.148131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.148238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.149523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.156152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 486/979] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:49.161481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.222099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.260093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaec9e-8165-4b4d-8c27-37c9f4416b6b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.274208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5c0da-b7b6-418d-b9aa-eeba6a978b9f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.295596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.296194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.296475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.296772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.297617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:49.297925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.299466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.308965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.309571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.309847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.310119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.310744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:49.311043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.312383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.317042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 487/980] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:49.319111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 494/981] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:49.323744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.324526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.344221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.428030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd548f9-72b5-40ae-a910-7c28621e0f37 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.431510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a98851-be98-448c-b8c4-9c6982dcac79 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.449536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.463329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.468693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.468693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.468693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.469455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.470123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.470407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.470684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.471396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] RBAC: Adding query filter params (interface=admin, service_id=4d7fa75eb78842339cbcdc072ce8419d, region_id=RegionOne) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:49.471709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.473274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.483677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 488/982] 10.222.0.27 () {56 vars in 1249 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/endpoints?service_id=4d7fa75eb78842339cbcdc072ce8419d&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:49.484499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.540557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.542054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.562097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.602110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11fd4-1439-46a7-b300-abe8d9aafd8b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.640497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.641442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.641803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.642070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.642681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.644007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.650898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 489/983] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:06:49.651567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 495/984] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:06:49.657652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.711805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.773563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2bb1-0be5-4942-9d8d-c3a76d7e5e6a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.812769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.813417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.814323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.814323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.826517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.827963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:49.844695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:49.845722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:49.846628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:49.846628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:49.846628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:49.846628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 490/985] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:49.851868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:49.954657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afeaa5-9627-4bda-8f7e-72e879bd317d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:49.972300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:49.990622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:49.991260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:49.991543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:49.991815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:49.992463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:49.992795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:49.994204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba367206-df20-4e5c-96a2-89e6f3805ea3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.006591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 496/986] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.012770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.109872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:50.110065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d65e0-dc49-449a-a17a-9db61a6e93b3 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.110297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d65e0-dc49-449a-a17a-9db61a6e93b3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.110585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d65e0-dc49-449a-a17a-9db61a6e93b3 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.111931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 497/987] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:50.116856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0e82-7eb1-4a39-a90d-0c514d7f8a4b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.117024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.117219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.117498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.135746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:50.137023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:50.155647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.156283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.156589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.156897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.164650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.166114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:50.177170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:50.178067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:50.183850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 491/988] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:50.184344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.216718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 498/989] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:50.235627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.289318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a449-84ca-40c2-8972-bf38ae9297eb None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.325195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.325803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.326085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.326355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.327041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:50.327355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.328803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.341073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 492/990] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.343629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe3478-fe4c-4756-a31d-6789044443ba None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.347774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.354010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:50.369505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:50.380169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.381106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.381393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.381859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.382252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.383618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:50.394032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:50.395062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:50.396499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 499/991] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:50.402584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.461115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f00f90-9c0c-4fd4-b868-1dbfab2645df None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.498231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.498836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.499109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.499378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.500103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] RBAC: Adding query filter params (role.id=875680c7c12b46e088ceeb8b0be7d817, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=76ca42843b2941e0860bca74f5d7c5f6) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:50.500410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.501856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.516703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 493/992] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/role_assignments?user.id=76ca42843b2941e0860bca74f5d7c5f6&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=875680c7c12b46e088ceeb8b0be7d817 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.516931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e792f8-45d9-4bb2-8c87-2b5e79764f23 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.555381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.556004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.556285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.556560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.557242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:50.557550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.558947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.564652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 500/993] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.571617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.666931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:50.687537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91601ade-a086-4f86-9d34-4368c1334357 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.726282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.726901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.727149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.727423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.728056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:50.728459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.729898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.735363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 494/994] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.742068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.781238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:50.847536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89d397-8747-4990-9860-8925d32e7d15 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:50.883148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:50.883764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:50.884053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:50.884330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:50.885065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] RBAC: Adding query filter params (interface=public, service_id=258f1cdc541144eaaebe2e4eda8ff655, region_id=RegionOne) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:50.885379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:50.886740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54fa356-ac52-4b69-bb59-f6e42250a510 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:50.891972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 501/995] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/endpoints?service_id=258f1cdc541144eaaebe2e4eda8ff655&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:50.897416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:50.997913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ad904-7580-4b04-bc19-4604327be1a0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:51.031633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:51.032244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:51.032524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:51.032830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:51.033439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:51.034797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7a96b-0a7b-49f6-a6aa-334b5a50e217 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:51.048697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 495/996] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:50 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:06:51.689198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fedd-1ec6-4801-b8ed-70efa976ac5d None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:51.689744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fedd-1ec6-4801-b8ed-70efa976ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:51.689744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2fedd-1ec6-4801-b8ed-70efa976ac5d None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:51.691121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 502/997] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:51.696643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:51.696965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:51.697241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:51.715218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:51.716403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:51.718453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:51.785473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 496/998] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:51.809577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2fedd-1ec6-4801-b8ed-70efa976ac5d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:51.907949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:51.912827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2fedd-1ec6-4801-b8ed-70efa976ac5d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:51.947274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:51.947912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:51.948172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:51.948436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:51.949038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:51.950390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:51.960003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:51.961067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 02:06:51.961984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:51.963466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 503/999] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Aug 8 02:06:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:51.967029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:52.067580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8a84-3650-4edb-9d22-66cae26a5faf None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:52.102816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:52.103426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.103703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.103970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.104593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:52.104920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:52.106269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:52.111131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 497/1000] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:52.117894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:52.228463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056cc677-7bed-47a0-9f1a-a23114ac7dbc None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:52.262467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:52.263082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.263362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.263636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.264270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:52.264577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:52.265983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dad61b-9897-4d41-9619-3e89f7e95272 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:52.276748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 504/1001] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Aug 8 02:06:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:52.277877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:52.280728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:52.378654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edbce3-3dcf-415f-83f1-0fc4f20d8202 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:52.412232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:52.412952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.413233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.413507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.414444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:52.415805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00907f0f-8424-4814-a3f8-759a9bae3535 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:52.488777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 498/1002] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:06:52.582474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd15d7-59bf-4df9-b001-da619fd2b1d0 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.582756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd15d7-59bf-4df9-b001-da619fd2b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.583033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd15d7-59bf-4df9-b001-da619fd2b1d0 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.585179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 505/1003] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:52.590813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.591102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.591377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.611057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:52.612263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:52.674714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:52.691716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 499/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:52.715693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd15d7-59bf-4df9-b001-da619fd2b1d0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:52.746576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:52.821521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd15d7-59bf-4df9-b001-da619fd2b1d0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:52.856545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:52.857207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:52.857488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:52.857760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] PATH_INFO: `/v3/services/image` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:52.858342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:52.859699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:52.868948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:52.869953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:52.870777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 506/1005] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Aug 8 02:06:52 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:52.875953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:52.977185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec119cca-3193-4637-afbf-2cef6906ae25 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.011447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.012042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.012328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.012604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.013280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:53.013589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.014955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcc03f-984a-4ec4-be69-7030fb25a9c4 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.024336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 500/1006] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:53.025609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:53.083488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937a9b-bae5-4d27-997d-86342b1e163e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.084188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937a9b-bae5-4d27-997d-86342b1e163e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.084465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c937a9b-bae5-4d27-997d-86342b1e163e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.086755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 501/1007] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:53.091415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.091705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.092009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.110131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:53.111325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:53.125457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:53.135205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a799-c32a-45b4-adeb-d01fc057c838 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.164259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:53.172450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.173088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.173365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.173635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.174493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:53.174656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.176058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.181426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 507/1008] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:53.185538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 502/1009] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:53.208575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:53.326058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40a3a-d0e5-4b68-a5a0-ac9b9eb7fdf6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.365128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.365845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.366124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.366395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.366968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.368327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:53.376908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:53.378178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:53.379324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 508/1010] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:53.384493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:53.494979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cec98-465f-431b-ba80-51322aed8a50 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.529997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.530610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.530890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.531162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.531791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:53.532098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.533528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.543425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 503/1011] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:53.544674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:53.568389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:53.659804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435fff4-bb71-4ec9-aa0c-ef78b7470f58 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.696741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.697355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.697632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.697901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.698532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:53.698845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.700193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8e583-1139-44dc-a14c-46996016e2e0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.709646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 509/1012] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Aug 8 02:06:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:53.711657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:53.763414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:53.820236 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834716-6b75-4d33-91c4-e9d01a6fcd3a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:53.854631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:53.855157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:53.855432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:53.855703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:53.856361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:53.857785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c063b0-adba-4121-af84-0f6661571c0f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:53.940686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 504/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:06:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:54.536242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce982-41c5-4020-8ea5-d36486b90490 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:54.536810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce982-41c5-4020-8ea5-d36486b90490 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:54.536890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce982-41c5-4020-8ea5-d36486b90490 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:54.538239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 510/1014] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:54.543581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:54.543871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:54.544150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:54.562291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:54.563467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:54.649137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 505/1015] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:54 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:54.671968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce982-41c5-4020-8ea5-d36486b90490 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:54.760562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:54.778571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce982-41c5-4020-8ea5-d36486b90490 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:54.813853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:54.814407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:54.814686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:54.815010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:54.816123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:54.817627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216ac3c-b09d-4d6b-b00c-0b52978d40ca None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:54.819850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 511/1016] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:54.825496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:54.852994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:54.934832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472865ba-96c9-4361-9708-0cf0fbc936c5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:54.969332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:54.969974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:54.970206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:54.970478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:54.971131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:54.972558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd7e1-2428-45b3-b707-9fe034698903 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:55.068776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 506/1017] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:54 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:55.231266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690080e-4af3-4b0c-98d2-a5a486be1e02 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:55.231549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690080e-4af3-4b0c-98d2-a5a486be1e02 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:55.231914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690080e-4af3-4b0c-98d2-a5a486be1e02 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:55.233606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 512/1018] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:55.238865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579fc3b-9108-4db6-b1b3-21a9fe745b6c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:55.239152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579fc3b-9108-4db6-b1b3-21a9fe745b6c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:55.239494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579fc3b-9108-4db6-b1b3-21a9fe745b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:55.259947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d579fc3b-9108-4db6-b1b3-21a9fe745b6c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:55.261159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d579fc3b-9108-4db6-b1b3-21a9fe745b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:55.332649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 507/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:55 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:55.354982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690080e-4af3-4b0c-98d2-a5a486be1e02 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:55.455429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690080e-4af3-4b0c-98d2-a5a486be1e02 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:55.489678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:55.490448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:55.490732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:55.491005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:55.491662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:55.493056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62f0b-099d-42cf-8abd-b7e86dce3dbf None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:55.596742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 513/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:06:55 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:06:56.057904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14452c-632b-46be-b602-84dc77770f8f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.058424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14452c-632b-46be-b602-84dc77770f8f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.058502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14452c-632b-46be-b602-84dc77770f8f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.059941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 508/1021] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:56.064944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.065236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.065509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.085286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:56.086481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:56.095211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:56.162482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 514/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:56.186164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14452c-632b-46be-b602-84dc77770f8f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:56.198794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:56.295555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14452c-632b-46be-b602-84dc77770f8f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:56.329829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:56.330380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.330659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.330931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.341942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:06:56.346796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:56.347720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:06:56.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:56.352833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 509/1023] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:06:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:56.354886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:56.455358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036939c-e3ab-492e-ab75-b585ac84040c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:56.488955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:56.490062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.490341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.490646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.491307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:56.491611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:56.493061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:56.498047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:56.506953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 515/1024] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:06:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:56.508191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:56.580106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:56.613712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705869d-1bcf-4b42-82ac-b6e44806e166 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:56.650223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:56.650909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.651190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.651470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.661953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:56.663602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:56.679179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:56.680396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:56.681344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:56.681344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:56.681344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:56.681344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 510/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:06:56.687485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:56.793608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713c4e7-23a6-4721-855b-b3e407a1b221 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:56.827392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:56.827999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.828279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.828553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.829225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:56.829530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:56.830940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:56.848212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 516/1026] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:56.849409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:56.950665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce950a-f199-4385-ab68-652700e6011b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:56.973102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:56.985199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:56.985833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:56.986111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:56.986382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:56.993727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:56.995154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:06:57.006949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:57.007927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:57.013823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 511/1027] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:06:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:57.014440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.075504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.116349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee872e-5f2c-4beb-8a7e-cd8ce71160dd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.151083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.151693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.151973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.152245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.153008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:57.153315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.154728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b122c-1ea8-4c64-830a-c858905ac6fd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.176698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 517/1028] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:57.179012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.195577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630dfd9-7402-4b75-a0c0-c648171b56cf None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.195863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630dfd9-7402-4b75-a0c0-c648171b56cf None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.196143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630dfd9-7402-4b75-a0c0-c648171b56cf None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.198028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 518/1029] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:06:57.202368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.202657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.202936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.221636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:57.222843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:57.291917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865822e-bebf-4780-955d-db140dc645eb None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.292896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 519/1030] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:06:57.316917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.326669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.327408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.327745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.328020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.369159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.378431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.397898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.399348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.421827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4aba-8099-47e9-8e6c-ec77b0b5b733 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.458814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.459456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.459751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.460010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.461162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.462602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.464337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 520/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:57.469733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.518287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 512/1032] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:06:57.519876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.576228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ed45-3eff-4102-8466-94a4f8898631 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.597342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.612073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.617752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.618543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.618982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.619464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.620276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.621906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126affa-c9c7-41e0-bbb8-54f4b7486f99 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.629845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da147538-125b-4c1a-b980-f12b5f73e64a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.664501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.665152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.665426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.665693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.675405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.676938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7848e867-b263-4d65-a4c5-096e9b4ab08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:06:57.690860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:57.691977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 513/1033] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9b999-3c8f-4512-8744-d15eba3559e5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9b999-3c8f-4512-8744-d15eba3559e5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9b999-3c8f-4512-8744-d15eba3559e5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.697717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 514/1034] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:57.698243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9b999-3c8f-4512-8744-d15eba3559e5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.736695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 521/1035] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:06:57 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:06:57.736695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.736695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.736695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.745200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:57.746506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:57.818639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9b999-3c8f-4512-8744-d15eba3559e5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.827281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 522/1036] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:57.850787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.861769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:57.862398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:57.862683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:57.862960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:57.863621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:57.863927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:57.865943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:57.867774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:57.885332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 515/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:57.885332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:57.969515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f70f1-395c-4083-bb81-286a3af214fc None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:57.977691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:58.001695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815cb8-28da-45aa-a4ab-1d4dea7cef59 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.005914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.006610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.006892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.007163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] PATH_INFO: `/v3/services/image` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.007756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.009284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:58.021092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:58.022088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:06:58.022939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 523/1038] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:58.028097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.040845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.041776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.042060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.042338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.052351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.053920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:06:58.065340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:06:58.066254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 516/1039] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:06:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:06:58.072565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.129607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:58.142008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75996dff-c4cf-477b-b61a-1229a331126e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.175358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.175953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.176227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.176489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.177948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:58.178257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.179614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.180861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7eca5-bddd-4fdb-be46-af9ae6afc8c1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.185623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 524/1040] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:58.191377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.215969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.216587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.216891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.217164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.217860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:58.218165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.219589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.224973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:58.237499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 517/1041] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:58.238548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.305613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbd47b-6d84-479c-9389-874535c691e9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.343851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400069e1-4fea-4ae1-9b87-4b4d38ffa101 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.344040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.344522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.344845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.345121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.345859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:58.346282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.347714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.357624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 525/1042] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:58.358570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.363620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:58.384833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.385477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.385758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.386039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.386814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] RBAC: Adding query filter params (role.id=875680c7c12b46e088ceeb8b0be7d817, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:58.387152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.388581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.393991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 518/1043] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=875680c7c12b46e088ceeb8b0be7d817 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:06:58.462862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:58.470276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28bcf-0bd3-4af8-b5a3-2b90849f2fc0 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.506699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.507327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.507607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.507883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.508604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] RBAC: Adding query filter params (interface=public, service_id=8cb611e0d9ef48938929d2721daf5a2d, region_id=RegionOne) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:06:58.508946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.510432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a800e1a-dede-48a5-b3a2-c7988f7b00cf None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.522200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 526/1044] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/endpoints?service_id=8cb611e0d9ef48938929d2721daf5a2d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:06:58.523928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:06:58.635139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45241fb5-cb32-4042-9cc0-f44613979436 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:06:58.669475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:06:58.670002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:58.670277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:58.670544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:58.671139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:06:58.672482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e7c09-1550-4d4a-bb12-bd1ee97d3754 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:06:58.679734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 519/1045] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:06:58 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:06:59.856022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c31-6e2c-4ad9-aec3-d0db64b03204 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:59.856512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c31-6e2c-4ad9-aec3-d0db64b03204 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:59.856512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e1c31-6e2c-4ad9-aec3-d0db64b03204 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:59.858005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 527/1046] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:06:59.862908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:06:59.863311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:06:59.863477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:06:59.882674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:06:59.883833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:06:59.921371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:06:59.953691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 520/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:06:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:06:59.978214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1c31-6e2c-4ad9-aec3-d0db64b03204 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.079138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e1c31-6e2c-4ad9-aec3-d0db64b03204 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.114906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.115623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.115895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.116162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.125988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:00.131690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:00.132618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:00.134002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:00.137025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 528/1048] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:00.138747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.240212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b650e4-998c-48cb-88cb-33e71f5941e6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.274875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.275482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.275756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.276020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.276698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:00.276999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:00.278349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec3ef-26a1-429e-9ec9-e65da42fa907 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:00.291574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 521/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:00.292993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.313798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.399928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8cb73-63aa-454b-ab8f-e9f4b5a56da5 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.413149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcee7-4979-4d74-9d83-344ed9b135bc None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.413428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcee7-4979-4d74-9d83-344ed9b135bc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.413981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcee7-4979-4d74-9d83-344ed9b135bc None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.415562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 522/1050] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:00.420373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.420691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.420969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.436190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.436853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.437131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.437399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] PATH_INFO: `/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.439268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:00.440612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:00.452286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] RBAC: Authorizing `identity:get_user(user_id=507b2e93d9274f1b98f5e4cbe0f5a01e)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:00.453761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:00.460151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 529/1051] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:00.461283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.475292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.520704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 523/1052] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:07:00.538237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.544244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.568380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bd761-5072-4c56-a179-cfc6ffa45d28 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.613889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.614595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.614866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.615191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.647175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8b54a-9bd9-41f2-9503-a401d3f073b6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.685024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.685357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:00.685575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.685850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.686116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.686762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a48d2-6e43-4226-b78d-eecf7960932f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:00.697431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-026c8f89-6dff-49f1-99e5-71f03432ec9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:00.702185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:00.703258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:00.704102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 524/1053] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:00.705402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7a761-3ecc-4b7e-ad4d-7e320a8f0f79 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.705402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7a761-3ecc-4b7e-ad4d-7e320a8f0f79 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.705402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7a761-3ecc-4b7e-ad4d-7e320a8f0f79 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.706349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 525/1054] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:00.710011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7a761-3ecc-4b7e-ad4d-7e320a8f0f79 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.735465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.764396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.789146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 530/1055] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:00.790534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.790825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.791101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.810138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:00.811334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:00.821397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7a761-3ecc-4b7e-ad4d-7e320a8f0f79 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.855285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:00.855898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:00.856179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:00.856456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:00.857152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:00.857462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:00.858831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:00.867394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 526/1056] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:00.868063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.883508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 531/1057] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:00.884341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:00.952122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:00.967192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a1098-55c3-4e9b-bd95-146f5b216ac6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.982652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd2193-ffb8-48e8-978b-af190619f782 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:00.990595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.002206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.007269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.007269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.007269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.007269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.007269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.007672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 527/1058] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:01.009524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.016940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.017574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.017894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.018176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.029570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.031216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:01.047492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:01.048623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:07:01.049920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:01.049920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:01.049920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:01.049920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 532/1059] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:01.049920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.113366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de777b-e243-4e0e-80a2-fb1e8dfca828 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.147556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.148177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.148467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.148744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] PATH_INFO: `/v3/services/image` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.149322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.149862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf4d3-9afc-4139-b69f-9a83ea133a70 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.150669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:01.159481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:01.160455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:01.167035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 528/1060] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:01.167878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.181889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.184354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.185002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.185284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.185558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.186277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=05c3ced6a6f04f29bd342b6d530cd353, user.id=507b2e93d9274f1b98f5e4cbe0f5a01e) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:01.186585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.188010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.193620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 533/1061] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/role_assignments?user.id=507b2e93d9274f1b98f5e4cbe0f5a01e&scope.project.id=05c3ced6a6f04f29bd342b6d530cd353&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:01.194397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.225262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.275440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ab14-6a26-4d6e-ba62-4e7094117fac None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.314263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.314884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.315161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.315431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.316091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:01.316379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.320410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.325824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530a555-462a-4898-b953-7a5c8c599182 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.332925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 529/1062] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:01.339880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.366668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.367680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.368133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.368582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.369643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] RBAC: Adding query filter params (name=image) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:01.370175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.371831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.377609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 534/1063] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:01.383345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.421548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.445446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875518d5-7229-4aea-bac5-99115618f5bc None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.477114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.483191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.483821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.484101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.488516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.492162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3e877-65cd-4846-9a78-d659b3f69f5f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.492526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.494025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:07:01.506358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:01.507544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:01.508514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 530/1064] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:01.513169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.531908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.532602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.532929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.533184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.533812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:01.534115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.535459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.545716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 535/1065] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:01.547391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.625892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcefd7-2ae9-4d37-93ba-2edd847e36fb None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.657336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902341-8e53-4b69-968b-cc243dc8f8f5 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.663217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.664302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.664763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.665184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.666076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:01.666386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.667886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.677494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:01.686762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 531/1066] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:01.693038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.702907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.703521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.703796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.704065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.704807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.706164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.769659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 536/1067] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:07:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:07:01.804349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916a35d-d2c5-4217-9a4c-17df651a34a1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:01.853176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:01.853834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:01.854111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:01.854377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:01.867160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:01.868689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:01.885946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:01.886974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:07:01.887862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:01.887862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:01.887862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:01.888595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 532/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:01.893475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:01.904301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:02.007135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c228e46-bae2-480a-bd45-802ec4b4554a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:02.042208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:02.042740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:02.043015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:02.043281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:02.043914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:02.044213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:02.045620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:02.058278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 537/1069] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:02.063965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:02.110816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:02.167965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757312ed-91fe-4ef4-b702-6bbbff83dc22 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:02.202376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:02.203001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:02.203278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:02.203551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:02.211242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:02.212713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:07:02.224437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:02.225347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:07:02.226257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:02.228739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 533/1070] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:07:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:02.231393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:02.300485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:02.338238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738a7f3-b07d-452b-8641-8d00b0fa936a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:02.373704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:02.374320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:02.374599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:02.374869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:02.375554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:02.375860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:02.377309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942841e5-1cdd-4d48-8c5b-377c1ebcd9c4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:02.393390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 538/1071] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:02.394775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:02.495914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40771-85f9-4be2-a336-2fa96226420b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:02.498337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:02.530310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:02.530990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:02.531298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:02.531633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/875680c7c12b46e088ceeb8b0be7d817` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:02.533777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=fcfff7acf53f4d779549a9688117f05d, role_id=875680c7c12b46e088ceeb8b0be7d817)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:02.535206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff5e92-e452-43b5-8b30-56d389095d0b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:02.606746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 534/1072] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:07:02 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/fcfff7acf53f4d779549a9688117f05d/roles/875680c7c12b46e088ceeb8b0be7d817 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:07:03.234082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee2f3-9af8-475c-8872-72757da6377f None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.234628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee2f3-9af8-475c-8872-72757da6377f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.234628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ee2f3-9af8-475c-8872-72757da6377f None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.236098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 539/1073] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:03.242067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.242506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.242732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.260386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:03.261656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:03.330894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 535/1074] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:03.354478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ee2f3-9af8-475c-8872-72757da6377f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:03.456278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ee2f3-9af8-475c-8872-72757da6377f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:03.482023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:03.491029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:03.491595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.491871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.492136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.501982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:03.507189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:03.508136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:03.509033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 540/1075] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:03.515829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:03.618307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:03.625534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abea42-b9a8-4da6-8cd0-c8381eb96946 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:03.662030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:03.662639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.662916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.663192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.664125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:03.664427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:03.665817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:03.674389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 536/1076] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:03.680707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:03.788320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c51666-a90c-4588-b62d-81227017cb66 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:03.822530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:03.823160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.823440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.823715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] PATH_INFO: `/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.824858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] RBAC: Authorizing `identity:get_user(user_id=507b2e93d9274f1b98f5e4cbe0f5a01e)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:03.826287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7f949-7864-43bc-8e28-01d120e5d93d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:03.832545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 541/1077] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:03 2022] GET /identity/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:03.834365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:03.853057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b196-d545-48c5-b0dc-a505e09128b2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.853389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b196-d545-48c5-b0dc-a505e09128b2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.853807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069b196-d545-48c5-b0dc-a505e09128b2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.855172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 542/1078] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:03.860567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.860909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.861183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.878593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:03.879759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:03.936466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:03.943293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1735d-1fb6-4ffd-bde0-512eaf171d58 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:03.956925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 543/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:03.979792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:03.980285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:03.981006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:03.981283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:03.981552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:03.982663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:03.984079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:03.985846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 537/1080] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:03 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:03.993276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:04.024148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:04.088681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee706a-ff07-4763-8049-145904afc7b0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:04.103938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb28294-29cf-45f0-8f4a-e88dd65cfc80 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:04.126155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:04.126768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.127056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.127327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.143096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:04.144047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.144545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.144916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353/users/507b2e93d9274f1b98f5e4cbe0f5a01e/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.147577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=05c3ced6a6f04f29bd342b6d530cd353, user_id=507b2e93d9274f1b98f5e4cbe0f5a01e, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:04.149034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:04.180158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:04.195935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:04.197434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:04.233775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 538/1081] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:07:03 2022] PUT /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353/users/507b2e93d9274f1b98f5e4cbe0f5a01e/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:07:04.307779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 544/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:07:04.313946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:04.404719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:04.422038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c181f8-1437-427a-b6bc-28aea4066ad7 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:04.456945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:04.457607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.457916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.458295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.468433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:04.469990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-69889d06-9d6d-4ab7-9cf5-1d117c0d54d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:04.485592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:04.486588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:07:04.487491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:04.487491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:04.487491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:04.487491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 539/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:04.493458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:04.578483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:04.603590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ec5c6-4b61-4665-b4a7-8683cf8f014a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:04.628872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b598b-df16-4fac-acc6-6694ea67a678 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.629182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b598b-df16-4fac-acc6-6694ea67a678 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.629437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b598b-df16-4fac-acc6-6694ea67a678 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.631170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 540/1084] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:04.635812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.636098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.636376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.638348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:04.638963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.639245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.639517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.640243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:04.640549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:04.641940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:04.660776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 545/1085] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:04.661482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:04.661482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:04.762896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 541/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:07:04.787152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:04.893945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887ecb-2941-48ca-b471-b2bb00674cb1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:04.929785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:04.930742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:04.931028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:04.931303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:04.942912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:04.947672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:04.948799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:04.949682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 546/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Aug 8 02:07:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:04.954832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.019552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:05.053400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fa85c-306a-4f90-ac09-a9244f14ac70 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.087690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.088297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.088575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.088876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.089535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:05.089842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.091212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:05.099200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 542/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:07:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:05.104787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.108495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:05.201571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59f43f8-8a6c-44eb-b962-e769d9498376 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.234121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.234741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.235020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.235292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.245474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.246907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:05.260814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:05.261824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:07:05.262733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:05.262733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:05.262733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:05.262733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 547/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:05.267631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.368367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662f19b-addf-4d98-be2a-e0c5e01b1407 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.395286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:05.403156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.403767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.407200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.407200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.407200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:05.407200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.407200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:05.418618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 543/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:05.424112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.483302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:05.529743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630877d-e61d-411e-8495-3c6994f24363 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.562680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.563301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.563584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.563854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.571905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.573375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:07:05.584318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:05.590064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:05.591788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 548/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Aug 8 02:07:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:05.592557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.693202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4f7b5-b2b4-40e1-8868-d2856c483fb3 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.727233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.727834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.728112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.728385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.729141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:05.729449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.730864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:05.747691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 544/1092] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Aug 8 02:07:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:05.753392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:05.852271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376327-36c6-4793-b20d-799c12162496 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:05.868052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:05.885810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:05.886465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:05.886746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:05.887020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:05.953944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:05.955409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.054031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 549/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:06.059606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.107007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.159281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326da396-1127-415e-93ed-1f1e698a8cdc None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.194689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.195307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.195582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.195850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.205688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.207109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-12b44cb8-82e5-4a7a-83dd-a7d41c985f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:06.222129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:06.223074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:07:06.228887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:06.228887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:07:06.228887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:06.228887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 545/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:06.229253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.246851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.272726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ae47-1970-4c4e-bac5-57a9371638ad None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.273008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ae47-1970-4c4e-bac5-57a9371638ad None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.273276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ae47-1970-4c4e-bac5-57a9371638ad None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.274608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 546/1095] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:07:06.279446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.279723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.280002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.298133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:06.299299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:06.329849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec25ac-d57b-4a27-ada5-8defb781b654 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.362977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.363584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.364386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.364785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.365436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:06.365740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.366207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 547/1096] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:07:06.369608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.385596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 550/1097] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:06.386644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.389432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.414684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.467186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.487413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410eac9-aa9b-4121-b05c-5092c7ac4a57 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.490088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c71bb-49be-4bfb-ad39-f88463c18d67 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.521640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.522261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.522535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.522802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.523379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.524001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.524283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.524560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.529320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.530747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.534268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:06.540844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:06.541851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:06.543118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 551/1098] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:07:06.543947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:06.544864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:06.545758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 548/1099] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:06.547830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.548782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.635985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.648261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c538e-089d-4e0e-bcf6-56e05980c202 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.650674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308818b2-ef7c-491b-ac40-527c0bae27ff None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.682727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.683333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.683607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.683874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.684557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:06.684893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.684988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.685579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.685860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.686132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.686337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.686827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:06.687133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.688485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.691026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.697936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 552/1100] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:06.702978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 549/1101] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:06.703637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.703956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.803483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e25542-ec2c-4c95-b444-de2eced36567 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.804697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790e3ac-bb0e-4925-8719-ef479e3a7ef6 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.837443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.837885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.838001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.838259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.838480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.838572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] PATH_INFO: `/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.838756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.839027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:06.839647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] RBAC: Authorizing `identity:get_user(user_id=507b2e93d9274f1b98f5e4cbe0f5a01e)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.839746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] RBAC: Adding query filter params (role.id=875680c7c12b46e088ceeb8b0be7d817, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=fcfff7acf53f4d779549a9688117f05d) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:06.840032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:06.841105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.841458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:06.842742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 553/1102] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/users/507b2e93d9274f1b98f5e4cbe0f5a01e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:07:06.848901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 550/1103] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/role_assignments?user.id=fcfff7acf53f4d779549a9688117f05d&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=875680c7c12b46e088ceeb8b0be7d817 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:06.849226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:06.918259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:06.953651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3041-431a-4381-a833-5e5bf4f34b3a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:06.990432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:06.991043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:06.991317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:06.991585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:07.059972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:07.061453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:07.125442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:07.166186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 554/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 02:07:07.172220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:07.188577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:07.274535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f63ad-7376-4f60-a03e-df62ebe8a67c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:07.312999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:07.313625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:07.313901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:07.314172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:07.315274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:07.316757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aee4e1-239c-4cc5-b55f-8e0159256659 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:07.318493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 551/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:07 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:07.325408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:07.427949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4a34c-2042-4b9d-861a-5299cdb32bd9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:07.461528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:07.462126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:07.462402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:07.462677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:07.463392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=05c3ced6a6f04f29bd342b6d530cd353, user.id=507b2e93d9274f1b98f5e4cbe0f5a01e) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:07.463695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:07.465152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421dcb3-cba4-47a7-b842-b0551cd66b8b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:07.470872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 555/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:07 2022] GET /identity/v3/role_assignments?user.id=507b2e93d9274f1b98f5e4cbe0f5a01e&scope.project.id=05c3ced6a6f04f29bd342b6d530cd353&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:09.530310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d72db-4065-4ae9-a055-762fd7e8e615 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:09.530875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d72db-4065-4ae9-a055-762fd7e8e615 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:09.530875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d72db-4065-4ae9-a055-762fd7e8e615 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:09.532179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 552/1107] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:09.537988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:09.538408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:09.538790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:09.557816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:09.559000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:09.581971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:09.625725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 556/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:07:09.650198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d72db-4065-4ae9-a055-762fd7e8e615 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:09.733172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:09.752469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d72db-4065-4ae9-a055-762fd7e8e615 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:09.786200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:09.786744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:09.787017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:09.787282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:09.788374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:09.790065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eae90-724f-4554-b7da-118f3923463c None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:09.791584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 553/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:09.797348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:09.894577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476abec-9030-40bc-87b3-3462a6006e0d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:09.929134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:09.929725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:09.930001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:09.930270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:09.930912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:09.932315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870034f-91c3-4513-b51e-e273f89f4ca9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:10.043177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 557/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:07:09 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:12.091557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd148-2374-4f20-a22a-14e52063d80c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:12.092134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd148-2374-4f20-a22a-14e52063d80c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:12.092134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd148-2374-4f20-a22a-14e52063d80c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:12.093459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 554/1111] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:12.099268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:12.099670 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:12.099809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:12.117390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:12.118559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:12.128786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:12.182782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 558/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:12.206302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dd148-2374-4f20-a22a-14e52063d80c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:12.306862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dd148-2374-4f20-a22a-14e52063d80c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:12.340217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:12.340850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:12.341125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:12.341392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] PATH_INFO: `/v3/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:12.342799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] RBAC: Authorizing `identity:get_role(role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:12.344085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:12.345401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 555/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:12 2022] GET /identity/v3/roles/47edb61814514a2b8e3e7b38f680c3da => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:12.351347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:12.449190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67b214-1978-4545-ac8c-1cc3f79f13ae None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:12.482282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:12.482901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:12.483185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:12.483461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] PATH_INFO: `/v3/users/cf4fed94759041c79bb41a5b707dc94a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:12.496762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fed94759041c79bb41a5b707dc94a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:12.498201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:12.499775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 559/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:12 2022] GET /identity/v3/users/cf4fed94759041c79bb41a5b707dc94a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:12.505525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:12.523851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:12.603686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c6b5-1005-4b54-874f-36d4f395f385 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:12.637043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:12.637632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:12.637908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:12.638176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:12.704553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:12.706009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:12.722275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:12.886563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 556/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:12.892440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:12.896380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:12.992481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c3f9c-f315-4ade-8e5c-488ab4f71fb9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:13.025913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:13.026536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:13.026818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:13.027099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:13.028216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:13.029735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1433fb5-8704-4051-abb4-127b67cae5c4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:13.031351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 560/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:12 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:13.037609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:13.135249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270bf3-7cf2-4da6-a543-c6246a6f7b08 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:13.151210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:13.168654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:13.169315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:13.169598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:13.169871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:13.170595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=05c3ced6a6f04f29bd342b6d530cd353, user.id=cf4fed94759041c79bb41a5b707dc94a) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:13.170911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:13.172326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a05bd-0e73-475e-afcb-514a2dee3b09 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:13.178089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 557/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:13 2022] GET /identity/v3/role_assignments?user.id=cf4fed94759041c79bb41a5b707dc94a&scope.project.id=05c3ced6a6f04f29bd342b6d530cd353&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:15.159332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec883-ecdc-4382-826d-e4901451a740 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.159880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec883-ecdc-4382-826d-e4901451a740 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.159880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec883-ecdc-4382-826d-e4901451a740 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.161344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 561/1118] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:15.166866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.168565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.168565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.186890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:15.188065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:15.252461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 558/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:15.275202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec883-ecdc-4382-826d-e4901451a740 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:15.344926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:15.374603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec883-ecdc-4382-826d-e4901451a740 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:15.408429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:15.409077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.409406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.409629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] PATH_INFO: `/v3/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.411572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] RBAC: Authorizing `identity:get_role(role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:15.412974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:15.414541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 562/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:15 2022] GET /identity/v3/roles/47edb61814514a2b8e3e7b38f680c3da => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:15.420611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:15.516598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fd892-2177-4f6d-820c-cb86a1c6e8e3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:15.527674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:15.550537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:15.551561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.551845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.552122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] PATH_INFO: `/v3/users/cf4fed94759041c79bb41a5b707dc94a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.553263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fed94759041c79bb41a5b707dc94a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:15.554699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:15.556273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 559/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:15 2022] GET /identity/v3/users/cf4fed94759041c79bb41a5b707dc94a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:15.561978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:15.658452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a8853-558d-418b-bb60-c87da0d8c043 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:15.690765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:15.691322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.692134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.692407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.693541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:15.694965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b092-a5f5-4647-b284-7cd91aa35520 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:15.696577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 563/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:15 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:15.702548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:15.799866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139539c-92e0-4d55-aaa3-4ce94beb8d95 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:15.832061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:15.833488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:15.833771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:15.834046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353/users/cf4fed94759041c79bb41a5b707dc94a/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:15.836165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] RBAC: Authorizing `identity:create_grant(project_id=05c3ced6a6f04f29bd342b6d530cd353, user_id=cf4fed94759041c79bb41a5b707dc94a, role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:15.837620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ede635-1823-4602-8f11-b161db8a5478 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:15.926964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 560/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:07:15 2022] PUT /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353/users/cf4fed94759041c79bb41a5b707dc94a/roles/47edb61814514a2b8e3e7b38f680c3da => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:07:17.902392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b2d9-4fb0-4629-aac3-9383b7a8c201 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:17.902905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b2d9-4fb0-4629-aac3-9383b7a8c201 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:17.902905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822b2d9-4fb0-4629-aac3-9383b7a8c201 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:17.904193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 564/1124] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:17.909336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:17.909754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:17.909888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:17.928860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:17.930044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:17.958546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:17.994468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 561/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:18.016759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822b2d9-4fb0-4629-aac3-9383b7a8c201 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:18.027724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:18.116469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822b2d9-4fb0-4629-aac3-9383b7a8c201 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:18.149266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:18.149889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:18.150173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:18.150449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] PATH_INFO: `/v3/roles/47edb61814514a2b8e3e7b38f680c3da` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:18.151832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] RBAC: Authorizing `identity:get_role(role_id=47edb61814514a2b8e3e7b38f680c3da)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:18.153285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:18.154533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 565/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:18 2022] GET /identity/v3/roles/47edb61814514a2b8e3e7b38f680c3da => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:18.160902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:18.256710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef851091-a1b4-46e9-94c5-ceafa8910e91 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:18.289970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:18.290584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:18.290857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:18.291126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] PATH_INFO: `/v3/users/cf4fed94759041c79bb41a5b707dc94a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:18.292220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] RBAC: Authorizing `identity:get_user(user_id=cf4fed94759041c79bb41a5b707dc94a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:18.293673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:18.295315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 562/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:18 2022] GET /identity/v3/users/cf4fed94759041c79bb41a5b707dc94a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:18.301551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:18.374618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:18.397386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad94342-554a-460c-854b-4d1c7cc022cd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:18.430543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:18.431067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:18.431343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:18.431610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:18.495128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:18.496625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:18.568006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:18.590638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 566/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:18.596630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:18.630513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:18.694000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b985e5c-d447-494e-bd98-fa1abfcd2002 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:18.727074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:18.727702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:18.727981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:18.728254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] PATH_INFO: `/v3/projects/05c3ced6a6f04f29bd342b6d530cd353` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:18.729375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=05c3ced6a6f04f29bd342b6d530cd353)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:18.730791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c9670-99c4-464c-b829-6122b9800bc2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:18.732401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 563/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:18 2022] GET /identity/v3/projects/05c3ced6a6f04f29bd342b6d530cd353 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:18.738620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:18.835402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf207-c044-4cc6-b2fe-e2f943b92675 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:18.869098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:18.869694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:18.869970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:18.870239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:18.870959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] RBAC: Adding query filter params (role.id=47edb61814514a2b8e3e7b38f680c3da, scope.project.id=05c3ced6a6f04f29bd342b6d530cd353, user.id=cf4fed94759041c79bb41a5b707dc94a) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:18.871266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:18.872711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeb39d-e3b0-4daa-b181-8aafdba1a664 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:18.878249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 567/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:18 2022] GET /identity/v3/role_assignments?user.id=cf4fed94759041c79bb41a5b707dc94a&scope.project.id=05c3ced6a6f04f29bd342b6d530cd353&role.id=47edb61814514a2b8e3e7b38f680c3da => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:20.883637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694ffe6-2bbb-4900-bd79-6cccbed8a2e6 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:20.884159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694ffe6-2bbb-4900-bd79-6cccbed8a2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:20.884159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694ffe6-2bbb-4900-bd79-6cccbed8a2e6 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:20.885549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 564/1131] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:20.890559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:20.890964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:20.891109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:20.910025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:20.911256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:20.944705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:20.976459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 568/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:20.999160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694ffe6-2bbb-4900-bd79-6cccbed8a2e6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:21.098973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694ffe6-2bbb-4900-bd79-6cccbed8a2e6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:21.101255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:21.131897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:21.132519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:21.132840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:21.133116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:21.134271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:21.135729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778c851-5fa2-47fc-86ee-b765e9e04a68 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:21.137304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 565/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:21.143062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:21.237806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e95b14-1202-4c2b-be3c-bf287027b4c9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:21.270550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:21.271148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:21.271429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:21.271696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:21.272330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:21.273788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f8b8e-6406-4f52-bf32-a85617746620 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:21.372321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 569/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:07:21 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:23.382108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ba11-f16a-499d-8298-35874b879b0e None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:23.382684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ba11-f16a-499d-8298-35874b879b0e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:23.382684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ba11-f16a-499d-8298-35874b879b0e None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:23.384004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 566/1135] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:23.389366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:23.389779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:23.389911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:23.407850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:23.409042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:23.425534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:23.476272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 570/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:23.502417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49ba11-f16a-499d-8298-35874b879b0e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:23.604816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49ba11-f16a-499d-8298-35874b879b0e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:23.638392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:23.638949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:23.639228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:23.639500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:23.640600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:23.642036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cfe5d-1e70-4932-8ebe-e74fb45b9596 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:23.643584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 567/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:23.649867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:23.745297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33521759-e008-4f65-8cc3-9f44d6a3e53a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:23.777466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:23.778060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:23.778341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:23.778613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:23.779259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:23.780687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0433c6-092c-4533-82e2-1fee62b1a045 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:23.881851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 571/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:07:23 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:25.918470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c196b3-1110-4158-88d7-c49c3ad7ae2b None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:25.918990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c196b3-1110-4158-88d7-c49c3ad7ae2b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:25.918990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c196b3-1110-4158-88d7-c49c3ad7ae2b None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:25.920262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 568/1139] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:25.926077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:25.926606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:25.926724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:25.944733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:25.945912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:25.947822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:26.010928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 572/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:26.033799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c196b3-1110-4158-88d7-c49c3ad7ae2b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:26.076465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:26.133757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c196b3-1110-4158-88d7-c49c3ad7ae2b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:26.166354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:26.166966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:26.167244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:26.167514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:26.178024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:26.182659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:26.183861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:26.184990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 569/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:26.190264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:26.290954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92058eb-12aa-4c52-8b63-fcb6fe1d82c7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:26.323975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:26.324579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:26.324894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:26.325171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:26.325835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:26.326142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:26.327533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:26.332995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:26.336241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 573/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:26.342069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:26.436738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:26.439482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cc003-dff7-40bb-ac6d-7bdbd65a6508 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:26.473178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:26.473791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:26.474065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:26.474335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] PATH_INFO: `/v3/users/8f688deb8500469a9ab33c1833fbfa6a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:26.486950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] RBAC: Authorizing `identity:get_user(user_id=8f688deb8500469a9ab33c1833fbfa6a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:26.488465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:26.490066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 570/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/users/8f688deb8500469a9ab33c1833fbfa6a => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:26.497816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:26.599492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a0ee2-2e55-43f4-ad1d-8b8d007d1ff7 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:26.631617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:26.632219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:26.632500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:26.632826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:26.696826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:26.698971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:26.700416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:26.795366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 574/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:26.801530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:26.900540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4aab05-b898-4895-9183-b0e9eb8d9e5a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:26.932810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:26.933428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:26.933705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:26.933979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] PATH_INFO: `/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:26.935086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] RBAC: Authorizing `identity:get_project(project_id=bc2e7eeaea644abfb759dbe10a32c6f6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:26.936505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bafca-e70e-4435-b342-c9c533d77f7b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:26.938150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 571/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:26.944851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:27.038657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:27.041369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3865-08a4-4960-8d69-58fbbe7d391a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:27.074534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:27.075125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:27.075405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:27.075677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:27.076392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=bc2e7eeaea644abfb759dbe10a32c6f6, user.id=8f688deb8500469a9ab33c1833fbfa6a) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:27.076726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:27.078145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d739d9-553e-447b-b6c1-f0cfb8b9b8b0 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:27.083326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 575/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:26 2022] GET /identity/v3/role_assignments?user.id=8f688deb8500469a9ab33c1833fbfa6a&scope.project.id=bc2e7eeaea644abfb759dbe10a32c6f6&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:29.083479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac764-6bbe-45d6-8bba-b32ebdada551 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.084050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac764-6bbe-45d6-8bba-b32ebdada551 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.084050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac764-6bbe-45d6-8bba-b32ebdada551 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.085398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 572/1147] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:29.090767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.091217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.091310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.109674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:29.110836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:29.178383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 576/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:29.200230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac764-6bbe-45d6-8bba-b32ebdada551 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:29.227360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:29.299250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac764-6bbe-45d6-8bba-b32ebdada551 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:29.332388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:29.333029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.333304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.333617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.343235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:29.348620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:29.349711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:29.350750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 573/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:29.355828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:29.445248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:29.451448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50390e2-9649-4e82-b069-a2a9a7e1ce9a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:29.485010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:29.485606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.485891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.486166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.486827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:29.487131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:29.488743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:29.496488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 577/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:29.502463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:29.587102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:29.602457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df84aed-f6f0-4ac7-b74f-646f4e34909a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:29.641722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:29.642340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.642618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.642893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] PATH_INFO: `/v3/users/8f688deb8500469a9ab33c1833fbfa6a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.644524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=8f688deb8500469a9ab33c1833fbfa6a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:29.645962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:29.647530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 574/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:29 2022] GET /identity/v3/users/8f688deb8500469a9ab33c1833fbfa6a => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:29.653435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:29.751509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ddd2e-c09b-41a2-9ec5-0ace488bbccc None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:29.784523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:29.785289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.785578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.785854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] PATH_INFO: `/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.786969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] RBAC: Authorizing `identity:get_project(project_id=bc2e7eeaea644abfb759dbe10a32c6f6)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:29.788634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9bb5b-bf5a-4714-8f5c-1656aec8257d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:29.790262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 578/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:29 2022] GET /identity/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:29.796043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:29.891722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f1142-7388-49c5-9352-36fbba14a5a5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:29.926019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:29.926651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:29.926930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:29.927202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] PATH_INFO: `/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6/users/8f688deb8500469a9ab33c1833fbfa6a/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:29.929095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] RBAC: Authorizing `identity:create_grant(project_id=bc2e7eeaea644abfb759dbe10a32c6f6, user_id=8f688deb8500469a9ab33c1833fbfa6a, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:29.930512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e19be-e21d-4433-b57a-cbc1b725921f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:30.045188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 575/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:07:29 2022] PUT /identity/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6/users/8f688deb8500469a9ab33c1833fbfa6a/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:07:32.044993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665057-72d5-4c54-8178-4c6d7431ac56 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.045550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665057-72d5-4c54-8178-4c6d7431ac56 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.045550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665057-72d5-4c54-8178-4c6d7431ac56 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.046877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 579/1154] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:32.053116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.053498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.053770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.071960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:32.073164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:32.082068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:32.138202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 576/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:32.161514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665057-72d5-4c54-8178-4c6d7431ac56 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:32.187099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:32.261187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665057-72d5-4c54-8178-4c6d7431ac56 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:32.294126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:32.294742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.295073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.295294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.305617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:32.310295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:32.311395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:32.312434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 580/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:07:32.317780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:32.413693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d11bf-7e7c-4cc9-8cea-ddcc3058adde None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:32.445849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:32.446453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.446734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.447009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.447668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:32.447973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:32.449350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:32.457799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 577/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:32.463395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:32.546257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:32.561372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4a32-b842-42b1-ae64-8d7e00c8a5fd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:32.594753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:32.595375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.595650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.595918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] PATH_INFO: `/v3/users/8f688deb8500469a9ab33c1833fbfa6a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.597114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] RBAC: Authorizing `identity:get_user(user_id=8f688deb8500469a9ab33c1833fbfa6a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:32.599166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:32.600742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 581/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:32 2022] GET /identity/v3/users/8f688deb8500469a9ab33c1833fbfa6a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:32.606550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:32.615786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:32.705458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3cbcf-6339-4554-8bb8-6854ff4fcede None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:32.737908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:32.738635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:32.738913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:32.739182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:32.803773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:32.805193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:32.813204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:32.902495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 578/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:32.908054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:33.012456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02937e86-6bbf-41c1-8fbe-bc9e6a352aaa None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:33.046929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:33.047568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:33.047825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:33.048102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] PATH_INFO: `/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:33.049243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] RBAC: Authorizing `identity:get_project(project_id=bc2e7eeaea644abfb759dbe10a32c6f6)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:33.051208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd5d87-01a1-4292-bcd0-693d53f60c7b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:33.052880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 582/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:32 2022] GET /identity/v3/projects/bc2e7eeaea644abfb759dbe10a32c6f6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:33.059099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:33.155788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4afaea-5df4-4a7a-a6b8-fb7867865963 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:33.163136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:33.189261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:33.189853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:33.190126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:33.190394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:33.191848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=bc2e7eeaea644abfb759dbe10a32c6f6, user.id=8f688deb8500469a9ab33c1833fbfa6a) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:33.192104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:33.193552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f38a2f-59ec-43bf-8ba6-356c7b0e31a3 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:33.199304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 579/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:33 2022] GET /identity/v3/role_assignments?user.id=8f688deb8500469a9ab33c1833fbfa6a&scope.project.id=bc2e7eeaea644abfb759dbe10a32c6f6&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:35.206411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50865-2438-4c2a-9563-45b69c77b658 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:35.206942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50865-2438-4c2a-9563-45b69c77b658 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:35.207131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf50865-2438-4c2a-9563-45b69c77b658 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:35.208481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 583/1162] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:35.213775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:35.214183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:35.214329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:35.232563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:35.233781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:35.298873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 580/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:35.321491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf50865-2438-4c2a-9563-45b69c77b658 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:35.343472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:35.420786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf50865-2438-4c2a-9563-45b69c77b658 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:35.452568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:35.453186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:35.453468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:35.453741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:35.517490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:35.519589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:35.540091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:35.615368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 584/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:35.621243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:35.714511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:35.717468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be3af-58a3-496a-880a-d5dce75b7006 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:35.749685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:35.750303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:35.750584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:35.750860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:35.758019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:35.759454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-7a63c094-97a7-47f8-b69c-fbc78ae60e40 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:35.770731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:35.771931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:35.772833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 581/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:07:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:35.777783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:35.873913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d436ddf-5d34-478a-94f0-4ca185981e5b None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:35.892966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:35.906712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:35.907316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:35.907598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:35.907872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:35.908512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:35.908848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:35.910830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc7b9-c628-4ccb-baa6-b71f90b8aeaa None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:35.918084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 585/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Aug 8 02:07:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:37.903064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e709-29af-4770-b80f-0086cb1624ce None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:37.903584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e709-29af-4770-b80f-0086cb1624ce None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:37.903584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165e709-29af-4770-b80f-0086cb1624ce None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:37.904913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 582/1167] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:37.910930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d876-a22e-4969-9685-77e339aa369c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:37.911310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d876-a22e-4969-9685-77e339aa369c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:37.911476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372d876-a22e-4969-9685-77e339aa369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:37.929591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2372d876-a22e-4969-9685-77e339aa369c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:37.930777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2372d876-a22e-4969-9685-77e339aa369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:37.994821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 586/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:38.017935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165e709-29af-4770-b80f-0086cb1624ce None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:38.114982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165e709-29af-4770-b80f-0086cb1624ce None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:38.148456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:38.149084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:38.149366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:38.149640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:38.150303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:38.151661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca20206-41fb-4961-b86b-f8625d422dd0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:38.241746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 583/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Mon Aug 8 02:07:38 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:40.295491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400385-845c-4c15-aa52-cf187cb1deb3 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:40.296072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400385-845c-4c15-aa52-cf187cb1deb3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:40.296072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400385-845c-4c15-aa52-cf187cb1deb3 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:40.297392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 587/1170] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:40.303133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:40.303512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:40.303689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:40.321455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:40.322632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:40.336247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:40.388981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 584/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:40.412393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77400385-845c-4c15-aa52-cf187cb1deb3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:40.509627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77400385-845c-4c15-aa52-cf187cb1deb3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:40.522814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:40.542373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:40.542981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:40.543257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:40.543526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] PATH_INFO: `/v3/domains/5899ca7f12ff43ae82c130e2ae226aeb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:40.554736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=5899ca7f12ff43ae82c130e2ae226aeb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:40.556096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9527ec-802f-4fef-bb1c-d50824f687fe None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:40.557678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 588/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Aug 8 02:07:40 2022] GET /identity/v3/domains/5899ca7f12ff43ae82c130e2ae226aeb => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:40.563518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:40.670766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa024ae1-518a-401d-9169-7e41f8d1eadd None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:40.703831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:40.704442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:40.704749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:40.705026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:40.705664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:40.707804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc9ef0-b8af-49ef-949a-0edc1eaa9a21 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:40.818822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 585/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:07:40 2022] POST /identity/v3/projects => generated 361 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:42.862719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193286-f11c-4e07-91e4-02988ee81215 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:42.863278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193286-f11c-4e07-91e4-02988ee81215 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:42.863278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193286-f11c-4e07-91e4-02988ee81215 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:42.864594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 589/1174] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:42.870420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:42.870897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:42.871002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:42.877400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:42.890026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:42.891219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:42.965512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 586/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:42.991347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193286-f11c-4e07-91e4-02988ee81215 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:43.093113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193286-f11c-4e07-91e4-02988ee81215 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:43.127476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:43.128691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:43.128990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:43.129346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] PATH_INFO: `/v3/domains/5899ca7f12ff43ae82c130e2ae226aeb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:43.130476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5899ca7f12ff43ae82c130e2ae226aeb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:43.131901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7c3a5-21b3-4e7d-bbc9-3e32a69f483e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:43.133983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 590/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Aug 8 02:07:42 2022] GET /identity/v3/domains/5899ca7f12ff43ae82c130e2ae226aeb => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:43.139655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:43.250982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb7ac4-b848-41a1-9d0e-f742e0c3e3da None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:43.280115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:43.285768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:43.286387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:43.286669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:43.286939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:43.287586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:43.289344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ffa1-a370-4086-914e-8c66c92f091a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:43.367681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 587/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:07:43 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:07:45.383377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becb61-da58-4f63-8cd9-bf4f138a60f1 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:45.383932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becb61-da58-4f63-8cd9-bf4f138a60f1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:45.383932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becb61-da58-4f63-8cd9-bf4f138a60f1 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:45.385268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 591/1178] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:45.390497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:45.390871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:45.391050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:45.409938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:45.411135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:45.475912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 588/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:45.499204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7becb61-da58-4f63-8cd9-bf4f138a60f1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:45.543213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:45.600654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7becb61-da58-4f63-8cd9-bf4f138a60f1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:45.636277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:45.636874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:45.637155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:45.637426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:45.647680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:45.652255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:45.653475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:45.654438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 592/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:45.659633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:45.756319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91298f93-fdd9-46c2-85db-647d2c6a3a62 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:45.773172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:45.789218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:45.789817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:45.790090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:45.790358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:45.791007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:45.791305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:45.792649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:45.800905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 589/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:45.806779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:45.902055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf41320-6d78-467f-91ef-e7b35b205000 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:45.903452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:45.935035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:45.935641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:45.935913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:45.936177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] PATH_INFO: `/v3/users/a91187291be74e2c9bcb680c4023c108` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:45.949227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] RBAC: Authorizing `identity:get_user(user_id=a91187291be74e2c9bcb680c4023c108)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:45.950661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:45.952229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 593/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:45 2022] GET /identity/v3/users/a91187291be74e2c9bcb680c4023c108 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:45.958014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:46.054048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14bc58-2c93-4bb3-be50-d09907dcb45d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:46.086009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:46.086595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:46.086868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:46.087135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:46.132623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:46.150752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:46.152240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:46.246314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 590/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:46.252323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:46.348030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25ad8e-293c-4eb2-a183-f6e44d959a0c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:46.380142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:46.380771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:46.381049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:46.381314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] PATH_INFO: `/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:46.382405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] RBAC: Authorizing `identity:get_project(project_id=23fc8e9c5c0e4075ae2a8b7cbdf7a113)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:46.383805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f02902-c53a-4a02-b027-59a6b928a785 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:46.385413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 594/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:46 2022] GET /identity/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:46.391384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:46.469731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:46.487825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86d3bc-f20b-41b9-9c5a-411b2b685c02 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:46.521190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:46.521779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:46.522060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:46.522334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:46.523056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=23fc8e9c5c0e4075ae2a8b7cbdf7a113, user.id=a91187291be74e2c9bcb680c4023c108) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:46.523366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:46.524816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d80c9-a947-4eea-842d-f664dfc96e71 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:46.529979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 591/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:46 2022] GET /identity/v3/role_assignments?user.id=a91187291be74e2c9bcb680c4023c108&scope.project.id=23fc8e9c5c0e4075ae2a8b7cbdf7a113&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:48.485831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bda96-e43a-481d-a26b-e9f25aaaf179 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:48.486381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bda96-e43a-481d-a26b-e9f25aaaf179 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:48.486381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bda96-e43a-481d-a26b-e9f25aaaf179 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:48.488076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 595/1186] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:48.493171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:48.496956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:48.496956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:48.512155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:48.513375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:48.577966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 592/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:48.600583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bda96-e43a-481d-a26b-e9f25aaaf179 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:48.628858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:48.700145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bda96-e43a-481d-a26b-e9f25aaaf179 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:48.733066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:48.733682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:48.733966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:48.734239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:48.744123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:48.748757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:48.749870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:07:48.750900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 596/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:48.756461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:48.830018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:48.853689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c872f8-dbb0-403e-b587-67dc982a81d3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:48.887480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:48.888086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:48.888366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:48.888641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:48.889338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:48.889644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:48.891021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:48.899906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 593/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:48.905737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:48.993082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:49.011793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6c204-f8dc-47c5-a437-462028cfd262 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:49.038668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:49.039273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:49.039555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:49.039822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] PATH_INFO: `/v3/users/a91187291be74e2c9bcb680c4023c108` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:49.040947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] RBAC: Authorizing `identity:get_user(user_id=a91187291be74e2c9bcb680c4023c108)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:49.042354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:49.043899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 597/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:48 2022] GET /identity/v3/users/a91187291be74e2c9bcb680c4023c108 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:49.050282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:49.145740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988698ff-1893-4b16-b2ab-6bb494f325de None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:49.177887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:49.178499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:49.178775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:49.179045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] PATH_INFO: `/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:49.180130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] RBAC: Authorizing `identity:get_project(project_id=23fc8e9c5c0e4075ae2a8b7cbdf7a113)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:49.181584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a51ee-f62d-4560-aa6d-66aa213a0084 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:49.183169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 594/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:49 2022] GET /identity/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:49.189146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:49.287903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282294d-bcc3-4cc4-8aa4-3f555b2fcf62 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:49.321225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:49.321861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:49.322137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:49.322407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] PATH_INFO: `/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113/users/a91187291be74e2c9bcb680c4023c108/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:49.324256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] RBAC: Authorizing `identity:create_grant(project_id=23fc8e9c5c0e4075ae2a8b7cbdf7a113, user_id=a91187291be74e2c9bcb680c4023c108, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:49.325705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c7cc4-406c-456b-9807-d9d593572adc None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:49.440172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 598/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:07:49 2022] PUT /identity/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113/users/a91187291be74e2c9bcb680c4023c108/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:07:51.407359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79ebb-ebce-4efc-b548-78a8cfbd82cb None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:51.407895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79ebb-ebce-4efc-b548-78a8cfbd82cb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:51.407895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79ebb-ebce-4efc-b548-78a8cfbd82cb None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:51.409297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 595/1193] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:07:51.415219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:51.415630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:51.415766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:51.434077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:07:51.435248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:07:51.445736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:51.499344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 599/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:07:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:07:51.523258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79ebb-ebce-4efc-b548-78a8cfbd82cb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:51.527148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:51.623900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79ebb-ebce-4efc-b548-78a8cfbd82cb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:51.656088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:51.656729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:51.657013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:51.657285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:51.668273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:07:51.672954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:07:51.674016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:07:51.676011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 596/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:07:51.681634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:51.778409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c033a7-b6a7-4bf0-93cf-4869c101f136 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:51.811905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:51.812494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:51.812801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:51.813070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:51.813719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:51.814017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:51.815986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:51.823604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 600/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:51.829306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:51.889151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:51.925847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac035c2-09b9-452c-90d0-3805d18b04f5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:51.959698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:51.960329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:51.960608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:51.960909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] PATH_INFO: `/v3/users/a91187291be74e2c9bcb680c4023c108` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:51.962028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] RBAC: Authorizing `identity:get_user(user_id=a91187291be74e2c9bcb680c4023c108)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:51.963465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:51.965093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 597/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Aug 8 02:07:51 2022] GET /identity/v3/users/a91187291be74e2c9bcb680c4023c108 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:51.970903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:51.982524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:52.068889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0398b-1838-4dd6-83a9-d5aad869c196 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:52.101764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:52.102353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:52.102633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:52.102905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:52.167792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:52.169257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:52.178454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:52.264119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 601/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:07:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:07:52.270152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:52.366710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5552d5b-d68b-49e4-b7fa-6fc9f3f0fb09 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:52.395443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:52.400093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:52.400732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:52.401008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:52.401275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] PATH_INFO: `/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:52.402372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] RBAC: Authorizing `identity:get_project(project_id=23fc8e9c5c0e4075ae2a8b7cbdf7a113)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:52.403798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a9f43-886e-4326-944c-829fa07a2a8b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:52.405438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 598/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Aug 8 02:07:52 2022] GET /identity/v3/projects/23fc8e9c5c0e4075ae2a8b7cbdf7a113 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:07:52.416729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:07:52.510972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b5428-a990-4e78-8476-7a2b9fec6c4c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:07:52.520931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:07:52.545189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:07:52.545777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:07:52.546088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:07:52.546354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:07:52.547061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=23fc8e9c5c0e4075ae2a8b7cbdf7a113, user.id=a91187291be74e2c9bcb680c4023c108) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:07:52.547363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:07:52.548806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c4093-8ddc-494a-a465-8b7c76319528 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:07:52.555323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 602/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:07:52 2022] GET /identity/v3/role_assignments?user.id=a91187291be74e2c9bcb680c4023c108&scope.project.id=23fc8e9c5c0e4075ae2a8b7cbdf7a113&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:08:02.570164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=14 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:10.508036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe7a2-ad03-4d4b-941f-924f87b083f0 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:10.508634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe7a2-ad03-4d4b-941f-924f87b083f0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:10.508634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe7a2-ad03-4d4b-941f-924f87b083f0 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:10.510074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 599/1201] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:10.515133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:10.515602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:10.515711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:10.533860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:10.544241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:10.603847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 603/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:10.628238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe7a2-ad03-4d4b-941f-924f87b083f0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:10.729717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe7a2-ad03-4d4b-941f-924f87b083f0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:10.763002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:10.763556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:10.763830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:10.764097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:10.774776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:10.776196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:10.790655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:10.792120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:10.797279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:10.797279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:10.797279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:10.797279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 600/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:10.798046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:10.897377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b907380-af9c-459f-bdfb-8d4cebb4f22c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:10.925408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:10.931321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:10.931897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:10.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:10.932438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:10.933094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:10.933392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:10.934741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea6cf8-007c-4a94-b6f9-1762831b4893 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:10.946351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 604/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:10.952502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:11.006868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:11.051916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7be-20e8-444f-a53c-6e36cf8a2aac None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:11.085178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:11.085776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:11.086052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:11.086320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:11.086957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:11.088383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96703845-7695-42f5-bcba-d5a1eb79a1fb None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:11.171000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 601/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Aug 8 02:09:10 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:09:13.236454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d008-d584-46b3-8369-eff91aaf8302 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.237049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d008-d584-46b3-8369-eff91aaf8302 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.237049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d008-d584-46b3-8369-eff91aaf8302 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.238405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 605/1206] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:13.244122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.244540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.244692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.262871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:13.264034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:13.330569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 602/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:13.355490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24d008-d584-46b3-8369-eff91aaf8302 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:13.467663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24d008-d584-46b3-8369-eff91aaf8302 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:13.502565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:13.503123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.503399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.503667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.514524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:13.519116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:13.525277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:13.526393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 606/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:09:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:13.527318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:13.566251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:13.624395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6a754-e4b8-40bc-8c8f-b9bebf698289 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:13.657237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:13.657840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.658112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.658397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.659051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:13.659349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:13.660706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:13.669528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 603/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:09:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:09:13.675398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:13.725839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:13.774426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a367b2-103f-42a2-b5fe-cce28d9cb9d2 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:13.809160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:13.809786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.810148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.810418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.820157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:13.821738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:13.835771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:13.836945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:13.838147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:13.838147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:13.838147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:13.838147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 607/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:13.842778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:13.940398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:13.943484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7e2b4-2f2f-4f39-8839-998a25f00e81 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:13.977150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:13.977749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:13.978024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:13.978290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:13.978921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:13.979219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:13.980587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:13.991522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 604/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:13.997226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:14.095597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ee977-f361-4d72-aa8a-38e42cd5d6b4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:14.097071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:14.128955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:14.129574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:14.129855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:14.130125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:14.137029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:14.138552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:14.150293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:14.151427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:14.152940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 608/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:14.157358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:14.261070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acea1fc-18cc-4bdb-95b9-119608cb78f2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:14.294950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:14.295551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:14.295830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:14.296102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:14.296892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:14.297200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:14.298626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:14.315467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 605/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:14.322027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:14.422481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae425f-0450-40f1-9795-df80b5ace0b3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:14.455966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:14.456724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:14.456927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:14.457199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:14.486680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:14.525080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:14.526586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:14.624361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 609/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:09:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:09:14.632615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:14.649696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:14.730234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050b602-fe60-4ba7-8afd-0668ee7a6899 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:14.764702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:14.765326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:14.765602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:14.765870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:14.775660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:14.782083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:14.791412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:14.792964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:14.794360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:14.794360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:14.794360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:14.794360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 606/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:09:14.798575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:14.858608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:14.897442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2efb3-646a-4a7f-ad3c-008d026df64e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:14.930679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:14.931286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:14.931591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:14.931870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:14.932513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:14.932853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:14.934234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:14.946441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 610/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:09:14.952054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:15.021030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:15.051036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804d269-ba0d-4205-841b-01e86c6866d5 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:15.085280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:15.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:15.086187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:15.086461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:15.092991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:15.099175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:15.106036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:15.107099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:15.108181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 607/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:15.113257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:15.214125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78802c07-a804-4d16-bf79-dc6d15d5b0d6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:15.237883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:15.249923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e964f-1550-4656-9130-598525594969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:15.250521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e964f-1550-4656-9130-598525594969 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:15.258624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e964f-1550-4656-9130-598525594969 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:15.258624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e964f-1550-4656-9130-598525594969 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:15.258624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e964f-1550-4656-9130-598525594969 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:15.258624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e964f-1550-4656-9130-598525594969 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:15.258624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e964f-1550-4656-9130-598525594969 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:15.264491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 611/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:15.270645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:15.370743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaead84-b180-437e-a00c-1ca9c22b0d98 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:15.398643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:15.404198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:15.404825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:15.405103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:15.405375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:15.406089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=46c46c81aeb14e189c21aa484404ad9e) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:15.406394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:15.407806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8f2d0-f59c-4615-873f-214dcb4a91ea None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:15.414170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 608/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:09:15 2022] GET /identity/v3/role_assignments?user.id=46c46c81aeb14e189c21aa484404ad9e&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:17.384420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a74b2-64b9-4090-a21b-639771aca302 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:17.385005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a74b2-64b9-4090-a21b-639771aca302 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:17.385005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6a74b2-64b9-4090-a21b-639771aca302 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:17.386347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 612/1220] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:17.391660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:17.392079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:17.392240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:17.410151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:17.411334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:17.477603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 609/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:17.501572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a74b2-64b9-4090-a21b-639771aca302 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:17.606879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6a74b2-64b9-4090-a21b-639771aca302 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:17.641041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:17.641579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:17.641856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:17.642123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:17.652257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:17.656880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:17.658043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:17.659070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 613/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:17.664923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:17.763305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c410da9-5d9b-4ccb-9dd9-20d833a0dcc8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:17.786863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:17.798693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:17.799330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:17.799610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:17.799881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:17.800541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:17.800881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:17.802263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:17.810076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 610/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:09:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:17.815890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:17.860419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:17.914750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd395a-c49f-4081-bb26-1d82f91b0f5c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:17.946836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:17.947437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:17.947714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:17.947994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:17.957464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:17.959459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:17.973098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:17.974210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:17.976100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:17.976100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:17.976100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:17.976100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 614/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:17.980445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.078993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b80639-fb96-4ac6-bad3-0180722333e9 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.113999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.114599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.114875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.115256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.116008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:18.116307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.117705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.129412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 611/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:18.135723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.229767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:18.236164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31479c8-df3d-4a7f-a5b0-8915f5946413 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.270311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.270936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.271230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.272319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.279187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.280759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:18.293132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:18.294080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:18.295151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 615/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:18.300147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.345824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:18.396043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd296b-ce2e-4ae9-a209-ceda5fd16872 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.428574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.429189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.429464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.429733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.430450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:18.430750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.432160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.449418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 612/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:18.455153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.555637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a38245-6344-4aa3-8ebd-ef065d6de99c None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.590137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.590852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.591134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.591407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.602233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.603736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:18.618230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:18.623811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:18.624962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:18.624962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:18.624962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:18.624962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 616/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:18.625259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.723409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb16bc-1d88-47dc-9810-f890eca3ae6e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.726146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:18.757438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.758033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.758307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.758576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.759211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:18.759511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.760888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.772095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 613/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:18.777831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:18.787150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:18.877097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bbd18-8560-4d84-a934-c4e47e3b73aa None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:18.909530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:18.910145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:18.910421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:18.910689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:18.918927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:18.920352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:18.932172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:18.933355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:18.934266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 617/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:18.939135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:19.038657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a261d-a4ca-4a4b-8c9b-d297de6bf8a2 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:19.072587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:19.073209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:19.073482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:19.073749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:19.074431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:19.074729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:19.076139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817f162-84b7-47a0-860e-f185756fdb9d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:19.088952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 614/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:19.094859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:19.160540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:19.196579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998dce-b6f2-443e-a612-9d5c8cef1001 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:19.232504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:19.233186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:19.233456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:19.233726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/46c46c81aeb14e189c21aa484404ad9e/roles/0c890622c50a4a56965f3278c48800ff` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:19.247919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=46c46c81aeb14e189c21aa484404ad9e, role_id=0c890622c50a4a56965f3278c48800ff)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:19.249377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f25d-eab1-44be-9460-f68c803d7007 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:19.406161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 618/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:09:19 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/46c46c81aeb14e189c21aa484404ad9e/roles/0c890622c50a4a56965f3278c48800ff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:09:21.383115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0b5b-2639-4497-b551-1b3ec49fb0bc None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:21.383646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0b5b-2639-4497-b551-1b3ec49fb0bc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:21.383646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da0b5b-2639-4497-b551-1b3ec49fb0bc None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:21.385208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 615/1233] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:21.390155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:21.390598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:21.390715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:21.409151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:21.410331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:21.476882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 619/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:21.500603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da0b5b-2639-4497-b551-1b3ec49fb0bc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:21.526012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:21.600727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da0b5b-2639-4497-b551-1b3ec49fb0bc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:21.634927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:21.635486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:21.635761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:21.636030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:21.645711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:21.650497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:21.651614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:21.652656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 616/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Aug 8 02:09:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:21.657652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:21.704746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:21.754988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7eee0-381e-4386-bb58-65e6741c5644 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:21.787921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:21.788519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:21.788829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:21.789104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:21.789795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:21.790131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:21.791506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:21.799345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 620/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Aug 8 02:09:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:21.805771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:21.895242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:21.910591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b5d6-db38-4039-a732-97240c21732f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:21.943595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:21.944211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:21.944492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:21.945212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:21.955260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:21.956751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:21.970420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:21.971522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:21.972642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:21.972642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:21.972642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:21.972642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 617/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:21.978063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:22.076177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9d3e5-9f0e-4b5f-899f-85d79e179c5f None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:22.084970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:22.109103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:22.109703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:22.109982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:22.110278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:22.110940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:22.111239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:22.112592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:22.124477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 621/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:22.130288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:22.230272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c72e4-acc5-4b30-9d3b-ca705535d59e None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:22.261944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:22.262563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:22.262841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:22.263107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:22.265736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:22.270595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:22.272557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:22.284302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:22.288582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:22.289817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 618/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:22.291841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:22.389936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3cee6-cc40-413a-b212-3a7dd66ab0dd None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:22.425129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:22.425744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:22.425989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:22.426257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:22.426975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:22.427274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:22.428821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:22.445441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 622/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:22.451244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:22.553674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57641b-7339-4b8e-b23c-02133f316b6a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:22.587359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:22.587958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:22.588232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:22.588500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:22.649283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:22.655433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:22.656955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:22.755090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 619/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:09:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:09:22.761399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:22.789656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:22.860478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b19af-aa66-49d4-9c9b-226f34fcbe5d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:22.893524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:22.894137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:22.894411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:22.894679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:22.904680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:22.906115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:22.921245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:22.922347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:22.923452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:22.923452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:22.923452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:22.923452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 623/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:22.928741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:23.022037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:23.027970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e15537-5aae-4ae3-9c1c-d0e1e6de5865 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:23.062737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:23.063338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:23.063615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:23.063915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:23.064552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:23.064880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:23.066233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:23.077034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 620/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:23.083098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:23.163591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:23.184816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eff7c-6d92-48a9-9d49-5d7ebd0883e9 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:23.217612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:23.218231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:23.218507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:23.218778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:23.225928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:23.227413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:23.238013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:23.239124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:23.240294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 624/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:23.245059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:23.343047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ccc37-520f-44bd-b38f-ddf597f15f81 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:23.375923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:23.376527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:23.376833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:23.377105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:23.377792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:23.378094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:23.379512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dd8c7-8c4a-418b-b59a-12f9a4338901 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:23.391621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 621/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:23.398287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:23.495666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2312d2-6a2c-4624-83d5-378f19e02154 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:23.528316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:23.528923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:23.529203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:23.529470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:23.530180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] RBAC: Adding query filter params (role.id=0c890622c50a4a56965f3278c48800ff, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=46c46c81aeb14e189c21aa484404ad9e) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:23.530481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:23.531889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a23ed-ef27-4958-9a73-c66d360ecac4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:23.537586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 625/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:09:23 2022] GET /identity/v3/role_assignments?user.id=46c46c81aeb14e189c21aa484404ad9e&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=0c890622c50a4a56965f3278c48800ff => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:25.603241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1df13-ccf4-497c-8c46-2cd51216672a None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:25.603702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1df13-ccf4-497c-8c46-2cd51216672a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:25.603809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1df13-ccf4-497c-8c46-2cd51216672a None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:25.605166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 622/1247] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:25.610328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:25.610750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:25.610944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:25.617979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:25.629721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:25.637161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:25.696649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 626/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:25.719965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1df13-ccf4-497c-8c46-2cd51216672a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:25.738988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:25.820264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1df13-ccf4-497c-8c46-2cd51216672a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:25.853536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:25.854169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:25.854443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:25.854711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:25.864798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:25.870203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:25.871322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:25.872350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 623/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:25.877471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:25.977492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913959fb-7520-460f-9c09-2603c88b79b5 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.006197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:26.010948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.011559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.011840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.012115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.012808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:26.013115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.014482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.022830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 627/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:09:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:26.028708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:26.105235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:26.127106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433e86-7eb1-467f-b173-3d1099c0a19c None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.160587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.161245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.161526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.161802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.173154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.174592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:26.189021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:26.190153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:26.191254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:26.191254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:26.191254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:26.191254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 624/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:09:26.196088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:26.294841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7fe0-f59d-40a3-9353-955c16a9f6eb None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.327725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.328335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.328618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.328922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.329559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:26.329862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.331231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.343288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 628/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:26.349076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:26.451839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d465b-ccd4-458c-a703-022b900a8c3a None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.478329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:26.486026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.486653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.486926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.487194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.494667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.496218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:26.507466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:26.513674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:26.514864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 625/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:26.515614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:26.570676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:26.614421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a140249-166a-40d5-959f-9cbaac66d3ec None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.650093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.650696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.650999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.651275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.652000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:26.652304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.653802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.671060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 629/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:26.677154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:26.777229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82645d37-2e8e-4f37-b16e-fc36b061dc27 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:26.810987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:26.811580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:26.811856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:26.812122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:26.866014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:26.878394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:26.879909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:26.978070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 626/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:09:26.983974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:27.082950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644a482-a547-4b5d-af85-c1ae45e876e3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:27.096450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:27.116270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:27.116922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:27.117197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:27.117468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:27.126951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:27.128418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:27.142739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:27.143937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:27.144900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:27.144900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:27.144900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:27.144900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 630/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:27.151747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:27.235596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:27.247549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1972c-0cc8-4999-91e7-be29e7d9977f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:27.281017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:27.281641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:27.281921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:27.282192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:27.282832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:27.283135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:27.284512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:27.295575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 627/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:27.301354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:27.400575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dc1fa-fc51-4a3c-b1fa-d5f09d05f611 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:27.432892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:27.433509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:27.433789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:27.434062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:27.440485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:27.441958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:27.453323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:27.458560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:27.459701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 631/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:27.461443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:27.561471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dc16c-02c2-4e79-af3f-9e3fca9763b8 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:27.594319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:27.594925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:27.595201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:27.595474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:27.596161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:27.596463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:27.597922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233518-a3d1-483e-9bf8-831bae36b787 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:27.609944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 628/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:27.616842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:27.642873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:27.715510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5d97-b115-49e3-9a66-255f2dbd672e None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:27.751966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:27.752577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:27.752892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:27.753170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:27.753940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=46c46c81aeb14e189c21aa484404ad9e) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:27.754216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:27.755640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38971428-763d-42d0-b3f5-e3da1ff8076f None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:27.761608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 632/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:09:27 2022] GET /identity/v3/role_assignments?user.id=46c46c81aeb14e189c21aa484404ad9e&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:29.742268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979968d-dcc9-4fc2-9079-45bb508f789f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:29.742790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979968d-dcc9-4fc2-9079-45bb508f789f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:29.742790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979968d-dcc9-4fc2-9079-45bb508f789f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:29.744145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 629/1261] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:29.749445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:29.749868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:29.749975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:29.769342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:29.770538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:29.838829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 633/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:29.861201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979968d-dcc9-4fc2-9079-45bb508f789f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:29.872642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:29.960588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979968d-dcc9-4fc2-9079-45bb508f789f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:29.993208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:29.993835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:29.994115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:29.994388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.005921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:30.011162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:30.012293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:30.013382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 630/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:09:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 02:09:30.018341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.036179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:30.112939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64152a-1239-4723-9d66-0d30d2be5678 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.147245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.147946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.148231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.148505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.149200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:30.149503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.150932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.158958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 634/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:30.165613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.232725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:30.265461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b6f5-2821-40e0-b814-7a6b8f752ab4 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.297709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.298327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.298605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.298876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.309793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.311312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:30.325103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:30.326069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:30.327324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:30.327324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:30.327324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:30.327324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 631/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:30.331968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.411955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:30.432232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa371fc-aeb9-4401-a5be-59056a0073c4 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.466973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.467579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.467859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.468132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.468801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:30.469105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.470459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.481821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 635/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:30.487497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.587947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6bc2-e633-4f00-9d85-e540a241633d None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.608891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:30.623067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.623694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.623968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.624233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.631438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.633389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:30.644180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:30.651044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:30.651973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 632/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:30.652554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.752406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc351ba-9643-4afb-802a-257e75e41503 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.786526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.787118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.787392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.787702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.789452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:30.789769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.791234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.800289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:30.808849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 636/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:30.814586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:30.911757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec185ff-74eb-4384-ac2b-508c7236c602 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:30.945113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:30.945716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:30.945990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:30.946258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:30.956211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:30.957662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:30.970997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:30.972114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:30.973367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:30.973367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:30.973367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:30.973367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 633/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:30.978314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:31.081140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1235b-656a-4988-b744-c4160fabbaa1 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:31.114916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:31.116222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:31.116222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:31.116222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:31.116851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:31.117158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:31.118508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:31.129389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 637/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:31.136144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:31.168476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:31.235383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97e90c-3d6a-413f-8937-40ff582aeb93 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:31.270528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:31.271175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:31.271436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:31.271755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:31.278385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:31.281460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:31.291369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:31.292407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:31.293750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 634/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:31.298644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:31.356971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:31.398617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55613ac9-ea2f-46cf-91fa-dbf3f0494562 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:31.432673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:31.433340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:31.433622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:31.433891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:31.434573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:31.434873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:31.436285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec55aef-ae25-4bd2-8d06-3e3c5d9bb7c8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:31.450310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 638/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:31.456276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:31.546322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:31.556200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf6419-72f6-44c1-93da-14b3768733f0 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:31.594322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:31.594322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:31.594322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:31.594322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] PATH_INFO: `/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/46c46c81aeb14e189c21aa484404ad9e/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:31.594702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] RBAC: Authorizing `identity:create_grant(project_id=7d51bb7c64ea44d0a65391702c0820ca, user_id=46c46c81aeb14e189c21aa484404ad9e, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:31.594702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963578f6-6826-40d8-aad3-1d124c303e3d None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:31.727329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 635/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Aug 8 02:09:31 2022] PUT /identity/v3/projects/7d51bb7c64ea44d0a65391702c0820ca/users/46c46c81aeb14e189c21aa484404ad9e/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:09:33.717422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3327d-3a99-4a69-9497-1e156aa80ae4 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:33.717945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3327d-3a99-4a69-9497-1e156aa80ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:33.717945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3327d-3a99-4a69-9497-1e156aa80ae4 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:33.719314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 639/1274] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:33.724610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:33.725025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:33.725229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:33.743889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:33.745156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:33.812315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 636/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:33.837285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3327d-3a99-4a69-9497-1e156aa80ae4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:33.938192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3327d-3a99-4a69-9497-1e156aa80ae4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:33.955858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:33.973863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:33.974480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:33.974756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:33.975024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:33.985108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 02:09:33.989703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:33.990807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:33.992053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 640/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Aug 8 02:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:33.997583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.079678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:34.100240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45077c-dd37-4bab-b482-5fc0fe729ffb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.129974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.130569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.130845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.131119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.131774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:34.132067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.134467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:34.142181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 637/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Aug 8 02:09:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:34.148480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.251653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c5ef-9a49-404d-bb42-37552c234bff None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.284555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.285198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.285477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.285739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.295503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.296985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:34.311621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:34.312622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:34.313660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:34.313660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:34.313660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:34.313660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 641/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:34.318440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.416839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad619-553a-4d42-9f32-d683f249917f None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.448812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.449401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.449674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.449940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.450565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:34.450861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.452205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:34.464105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 638/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:34.469983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.500728 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:34.568978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64c1d7-2bf5-40fc-8a7d-b01c7193ea5a None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.605181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.605868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.606176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.606446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.613277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.614697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 02:09:34.626252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:34.627228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:34.628447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 642/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:34.633718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.638110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:34.732956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4c5c-c9af-4563-af82-ed52edc2c933 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.765492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.766087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.766363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.766632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.767350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:34.767653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.769113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:34.787129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 639/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Aug 8 02:09:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:34.793882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:34.884214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:34.893302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ad21-f52b-4c31-9ab5-c6b4d89b542d None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:34.926448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:34.927048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:34.927326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:34.927593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:34.992284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:34.993702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.087564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 643/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Aug 8 02:09:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 02:09:35.089974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:35.093345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:35.153915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:35.191172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6c690-34c4-4f76-88b4-9510cdba07f1 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:35.223906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:35.224524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:35.224831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:35.225107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:35.235450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:35.236910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 02:09:35.253858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:35.255103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 02:09:35.256195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 02:09:35.256195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 02:09:35.256195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:35.256195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 640/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:35.258829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:35.355711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e14b6e-0aa1-4ea2-8724-e5baed2e3722 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:35.389164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:35.389757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:35.390034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:35.390304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:35.390987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:35.391283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:35.392629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.404225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 644/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Aug 8 02:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:35.409849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:35.506018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fca587-1471-4a0d-bccf-2fdd92ecff8b None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:35.519633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:35.539558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:35.540297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:35.540600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:35.540905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:35.547795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:35.549298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 02:09:35.560115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:35.561228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: ERROR keystone.server.flask.application Aug 08 02:09:35.562469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 641/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Aug 8 02:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:35.567226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:35.617948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:35.665810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96c488-0a7c-4160-835c-e6b792520db6 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:35.698883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:35.699478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:35.699756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:35.700029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:35.700735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:35.701043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:35.702458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10e41c-e59c-4658-a593-a8aabcf871a4 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.714113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 645/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Aug 8 02:09:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:35.720170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:35.818390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29720a3f-4714-4d24-91ee-8d62eafde867 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:35.851097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:35.852111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:35.852394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:35.852698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:35.853447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] RBAC: Adding query filter params (role.id=c555140ef854486a808b99fb021d4982, scope.project.id=7d51bb7c64ea44d0a65391702c0820ca, user.id=46c46c81aeb14e189c21aa484404ad9e) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:35.853769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:35.855175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3043a-0444-4082-bb72-049f16c57ca8 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:35.860838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 642/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Aug 8 02:09:35 2022] GET /identity/v3/role_assignments?user.id=46c46c81aeb14e189c21aa484404ad9e&scope.project.id=7d51bb7c64ea44d0a65391702c0820ca&role.id=c555140ef854486a808b99fb021d4982 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:37.880844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675d2-6e2a-480f-8c22-493deb53e78c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:37.881388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675d2-6e2a-480f-8c22-493deb53e78c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:37.881388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949675d2-6e2a-480f-8c22-493deb53e78c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:37.882679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 646/1288] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:37.888109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:37.888543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:37.888654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:37.907000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:37.908182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:37.938030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:37.973707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 643/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:37.997349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949675d2-6e2a-480f-8c22-493deb53e78c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:38.096784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949675d2-6e2a-480f-8c22-493deb53e78c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:38.108776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:38.131219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:38.131780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:38.132064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:38.132440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] PATH_INFO: `/v3/services/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:38.133145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:38.134622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:38.143145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:38.144351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:38.145326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 647/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:09:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:38.151010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:38.249684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7052-7afe-45bd-8df3-6d0db1eeceac None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:38.285277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:38.285886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:38.286185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:38.286439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:38.287073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:38.287377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:38.288917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f0e82-5178-4ce0-8034-5001068f5030 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:38.295157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 644/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Aug 8 02:09:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:38.301025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:38.398737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232df2-1e59-487a-83e5-2ddae3b2e8da None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:38.432425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:38.433092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:38.433369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:38.433638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] PATH_INFO: `/v3/services` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:38.434263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:38.434565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:38.435975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19c2e4-8d2b-4c49-816e-7bb6e90563b8 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:38.440428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 648/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Aug 8 02:09:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:40.426317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227305a7-f1cd-4d36-87fa-e4c432bb061c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:40.426854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227305a7-f1cd-4d36-87fa-e4c432bb061c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:40.426854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227305a7-f1cd-4d36-87fa-e4c432bb061c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:40.428287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 645/1293] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:40.433502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17c59-e9ce-4bca-877a-a970ee702261 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:40.433929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17c59-e9ce-4bca-877a-a970ee702261 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:40.434042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17c59-e9ce-4bca-877a-a970ee702261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:40.451784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5af17c59-e9ce-4bca-877a-a970ee702261 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:40.452984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5af17c59-e9ce-4bca-877a-a970ee702261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:40.482995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:40.519124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 649/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 02:09:40.544336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227305a7-f1cd-4d36-87fa-e4c432bb061c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:40.575353 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:40.644995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227305a7-f1cd-4d36-87fa-e4c432bb061c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:40.678744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:40.679288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:40.679562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:40.679829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:40.680929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:40.682297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a8b40-60d6-440e-8ddc-b5a14099c2ad None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:40.801373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 646/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Aug 8 02:09:40 2022] POST /identity/v3/services => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:09:42.885543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6c5b-c7c2-44b4-b0a0-af817f84ad20 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:42.886088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6c5b-c7c2-44b4-b0a0-af817f84ad20 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:42.886088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc6c5b-c7c2-44b4-b0a0-af817f84ad20 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:42.887379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 650/1296] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:42.892894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:42.893414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:42.893414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:42.911908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:42.913261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:42.988851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 647/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:43.013517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc6c5b-c7c2-44b4-b0a0-af817f84ad20 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:43.118949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc6c5b-c7c2-44b4-b0a0-af817f84ad20 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:43.154313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:43.154967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:43.155479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:43.155764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:43.156353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:43.157762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:43.166592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:43.167809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:43.168880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 651/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:09:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:43.174154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:43.193463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:43.278125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f5ab0-da01-4a0d-b932-983a611977e0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:43.312956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:43.313559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:43.313836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:43.314106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:43.314731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:43.315032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:43.316418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:43.322020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 648/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Aug 8 02:09:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:43.328605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:43.360419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:43.428100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00efa-22fc-44cf-a79b-9dd88506ade3 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:43.464114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:43.464740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:43.465022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:43.465291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:43.465988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] RBAC: Adding query filter params (interface=public, service_id=db1fb877003640d98335396092554729, region_id=RegionOne) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:43.466298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:43.467658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa4117-9590-4685-bb45-763e8e701e87 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:43.472630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 652/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Aug 8 02:09:43 2022] GET /identity/v3/endpoints?service_id=db1fb877003640d98335396092554729&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:43.479277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:43.563024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:43.589100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c648be-c303-449d-aded-518a397b4972 None admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:43.636260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:43.636899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:43.637175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:43.637441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:43.638042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:43.639393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e64c4-fc72-45ea-b0a2-4577bad170be None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:43.646147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 649/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Aug 8 02:09:43 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:09:45.664072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b7cd3-4a46-4135-b2d0-24b6abf31326 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:45.664707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b7cd3-4a46-4135-b2d0-24b6abf31326 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:45.664707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b7cd3-4a46-4135-b2d0-24b6abf31326 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:45.666052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 653/1302] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:09:45.671383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860eba0-5433-4378-a255-af09ef718509 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:45.671786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860eba0-5433-4378-a255-af09ef718509 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:45.671939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860eba0-5433-4378-a255-af09ef718509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:45.690662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7860eba0-5433-4378-a255-af09ef718509 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:09:45.695042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7860eba0-5433-4378-a255-af09ef718509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:09:45.758114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 650/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:09:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 02:09:45.781377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b7cd3-4a46-4135-b2d0-24b6abf31326 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:45.875369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:45.881785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b7cd3-4a46-4135-b2d0-24b6abf31326 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:45.915651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:45.916283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:45.916561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:45.916867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:45.917443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:45.918793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 02:09:45.927328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 02:09:45.928813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application with Lock( Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self._enter() Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application return self.creator() Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application created_value = creator( Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: ERROR keystone.server.flask.application Aug 08 02:09:45.929970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 654/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Aug 8 02:09:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 02:09:45.935420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860eba0-5433-4378-a255-af09ef718509 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:46.010154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:46.032342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860eba0-5433-4378-a255-af09ef718509 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:46.066846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:46.067442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:46.067715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:46.067982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] PATH_INFO: `/v3/services` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:46.068637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:09:46.068966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:46.070317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1006af7-9214-4ebe-8cac-bfbf9fde8cd9 None admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:46.075439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 651/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Aug 8 02:09:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:09:46.081345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:09:46.182113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a336abd-544b-45d6-a7bf-b424be911805 None admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:09:46.215326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:09:46.215929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:09:46.216209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:09:46.216482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:09:46.217389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:09:46.218744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f2033-e694-4a88-9d2d-19ecd9601e2e None admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:09:46.309586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 655/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:09:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:09:56.080838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=30 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:09:56.305014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:19.246521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828306a3-9eb7-4dcc-a3cc-2a4c0fba51ba None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:19.252951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828306a3-9eb7-4dcc-a3cc-2a4c0fba51ba None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:19.252951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828306a3-9eb7-4dcc-a3cc-2a4c0fba51ba None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:19.255328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dc60-9fa7-4e1d-8d39-8bec5b4c2f2e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:19.256249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dc60-9fa7-4e1d-8d39-8bec5b4c2f2e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:19.256770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5dc60-9fa7-4e1d-8d39-8bec5b4c2f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:19.261884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 652/1307] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:19 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:19.391903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1df5dc60-9fa7-4e1d-8d39-8bec5b4c2f2e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:19.397503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1df5dc60-9fa7-4e1d-8d39-8bec5b4c2f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:19.558120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 656/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:19.614450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:19.615549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:19.616012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:19.618678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 653/1309] 10.222.0.27 () {54 vars in 965 bytes} [Mon Aug 8 02:17:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:19.626307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c76232-09bd-4124-8216-8b8210bb9ebd None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:19.627209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c76232-09bd-4124-8216-8b8210bb9ebd None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:19.627868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c76232-09bd-4124-8216-8b8210bb9ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:19.722941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-66c76232-09bd-4124-8216-8b8210bb9ebd None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:19.724158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-66c76232-09bd-4124-8216-8b8210bb9ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:19.749497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:19.808774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 657/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Mon Aug 8 02:17:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:19.818352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:19.820159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:19.940619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a860c-fbc3-4a9c-b9bf-06da0c320266 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:20.019337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:20.020516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:20.020952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:20.021278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:20.022896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:20.080218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:20.135199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:20.136646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd9799-342f-4755-9d23-47627337fe81 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:20.159620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 654/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:17:22.478663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:22.479204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:22.479204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:22.480678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 658/1312] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:22.486075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5180b-de31-4e8c-976c-2eb516fd4a32 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:22.486619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5180b-de31-4e8c-976c-2eb516fd4a32 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:22.486619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5180b-de31-4e8c-976c-2eb516fd4a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:22.580201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d8a5180b-de31-4e8c-976c-2eb516fd4a32 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:22.584300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d8a5180b-de31-4e8c-976c-2eb516fd4a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:22.692822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 655/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:22.731773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:22.733294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:22.848360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97dad3-f455-4893-8631-087b7f822912 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:22.928339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:22.931216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:22.931832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:22.932364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:22.932773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:22.934397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:23.049028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:23.051117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42622e5-8027-4df3-8d4c-5600e0803926 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:23.069904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 659/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:25.586564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:25.586929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:25.587075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:25.588422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 656/1315] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:25.593563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d3fa0-f361-43d5-9ae3-d7d185fdc4db None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:25.594009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d3fa0-f361-43d5-9ae3-d7d185fdc4db None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:25.594105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d3fa0-f361-43d5-9ae3-d7d185fdc4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:25.659412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e43d3fa0-f361-43d5-9ae3-d7d185fdc4db None None] MFA Rules not processed for user `a1af2ddf4f644c50b16fc01349ed03ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:25.661440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e43d3fa0-f361-43d5-9ae3-d7d185fdc4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:25.716730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:25.770851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 660/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:25.815272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:25.817589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:25.827667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:25.919631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7b3ea-2a22-4c8b-933b-30143b7480b3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:25.990045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:25.990646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:25.990925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:25.991196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:25.992812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:26.054336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:26.100046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:26.101494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ffe5-bca1-4fe0-a2e6-98be3de84abb service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:26.119443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 657/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:29.062345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76262a1-f1d7-4cd5-b6d2-b34ba3882f5e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:29.062895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76262a1-f1d7-4cd5-b6d2-b34ba3882f5e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:29.062895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76262a1-f1d7-4cd5-b6d2-b34ba3882f5e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:29.064246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 661/1318] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:29.069773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4d6a-2fcb-4069-9f70-e366e7bcf81d None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:29.070175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4d6a-2fcb-4069-9f70-e366e7bcf81d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:29.070323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4d6a-2fcb-4069-9f70-e366e7bcf81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:29.133829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bb8c4d6a-2fcb-4069-9f70-e366e7bcf81d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:29.135026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bb8c4d6a-2fcb-4069-9f70-e366e7bcf81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:29.231112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 658/1319] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:29.282974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:29.283258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:29.283567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:29.284972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 662/1320] 10.222.0.27 () {54 vars in 965 bytes} [Mon Aug 8 02:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:29.291016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfea9-7652-453c-9015-e87074e8dff9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:29.291304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfea9-7652-453c-9015-e87074e8dff9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:29.291582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfea9-7652-453c-9015-e87074e8dff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:29.313775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:29.366589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b12cfea9-7652-453c-9015-e87074e8dff9 None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:29.367802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b12cfea9-7652-453c-9015-e87074e8dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:29.442844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 659/1321] 10.222.0.27 () {58 vars in 1062 bytes} [Mon Aug 8 02:17:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:29.451806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:29.453376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:29.556363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620e990-8f09-4ba4-83c3-dd39601c53d4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:29.631305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:29.631914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:29.632196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:29.632472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:29.634113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:29.640920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:29.742818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:29.744249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9667ac-fa0c-457f-ab21-12a29d764000 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:29.762228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 663/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:33.456516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:33.457083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:33.457211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:33.458579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 660/1323] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:33.463812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9352a-8b5d-4e4f-9289-9e20d48b96e4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:33.464248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9352a-8b5d-4e4f-9289-9e20d48b96e4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:33.464349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9352a-8b5d-4e4f-9289-9e20d48b96e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:33.533327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a2d9352a-8b5d-4e4f-9289-9e20d48b96e4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:33.535215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a2d9352a-8b5d-4e4f-9289-9e20d48b96e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:33.602599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:33.644536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 664/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:17:33.681707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:33.683524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:33.788626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb1ab1-6d36-49d6-a6b2-e084688b7072 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:33.800661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:33.862016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:33.862638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:33.862914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:33.863178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:33.864800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:33.975935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:33.977325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9532-2304-4683-a0ce-92273d8ad300 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:33.995456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 661/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:36.090656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:36.091195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:36.091195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:36.092464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 665/1326] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:36.097669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6552-a36f-488c-af60-a584009c2a3a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:36.098083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6552-a36f-488c-af60-a584009c2a3a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:36.098347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6552-a36f-488c-af60-a584009c2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:36.148386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:36.167720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2edf6552-a36f-488c-af60-a584009c2a3a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:36.168962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2edf6552-a36f-488c-af60-a584009c2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:36.267782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 662/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:36.305430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:36.307340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:36.412414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd0e47-3718-40b1-bd05-a00b6d0b635e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:36.492318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:36.492979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:36.493264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:36.493539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:36.495182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:36.512573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:36.609960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:36.611414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b77c-b77c-4331-ad73-193fcb7745b0 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:36.637656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 666/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:38.699405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:38.699969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:38.699969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:38.701324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 663/1329] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:38.706558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2a7e-55c5-49e9-953a-1b075cdf9de1 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:38.707011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2a7e-55c5-49e9-953a-1b075cdf9de1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:38.707320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a2a7e-55c5-49e9-953a-1b075cdf9de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:38.774636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5f7a2a7e-55c5-49e9-953a-1b075cdf9de1 None None] MFA Rules not processed for user `a1af2ddf4f644c50b16fc01349ed03ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:38.775909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5f7a2a7e-55c5-49e9-953a-1b075cdf9de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:38.842395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:38.879879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 667/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:38.919110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:38.920656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:39.025084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7181a-80c9-4e4a-8e38-93ff78fc5fc5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:39.045258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:39.101858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:39.102473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:39.102777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:39.103051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:39.104717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:39.212461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:39.213943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19942cc0-fed1-498d-b881-f7303e228a3e service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:39.232384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 664/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:41.760773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:41.761203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:41.761299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:41.762692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 668/1332] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:41.767849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330079-9555-48dc-bde5-5a26afd6bca1 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:41.768263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330079-9555-48dc-bde5-5a26afd6bca1 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:41.784964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330079-9555-48dc-bde5-5a26afd6bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:41.832345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:41.834929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b1330079-9555-48dc-bde5-5a26afd6bca1 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:41.836146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b1330079-9555-48dc-bde5-5a26afd6bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:41.932001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 665/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:41.967834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:41.969419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:42.079713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b427765-a818-4624-8417-1dc9b177d954 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:42.153322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:42.153927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:42.154208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:42.154483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:42.156077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:42.184322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:42.264560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:42.266029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06553e0-06f9-4999-a49f-bbb791b329e3 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:42.286389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 669/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:44.340597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.341165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.341165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.342521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 666/1335] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:44.347600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4b86-1bd1-43da-9f21-d101adcf937d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.348033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4b86-1bd1-43da-9f21-d101adcf937d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.348152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a4b86-1bd1-43da-9f21-d101adcf937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.412751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-496a4b86-1bd1-43da-9f21-d101adcf937d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:44.413963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-496a4b86-1bd1-43da-9f21-d101adcf937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:44.486812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:44.510045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 670/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:44.550636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:44.552173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:44.653902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579f647-5b8f-475d-9e01-78ff44f4ae57 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:44.682722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:44.727220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:44.727820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:44.728097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:44.728362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:44.729963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:44.836917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:44.838339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9fe72-6307-4f51-9bec-b986df32ed7e service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:44.856910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 667/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:47.530661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:47.531203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:47.531203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:47.532554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 671/1338] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:47.537733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313159f5-fe3b-4727-ace9-b6714e0a702a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:47.538173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313159f5-fe3b-4727-ace9-b6714e0a702a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:47.538288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313159f5-fe3b-4727-ace9-b6714e0a702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:47.603795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-313159f5-fe3b-4727-ace9-b6714e0a702a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:47.605053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-313159f5-fe3b-4727-ace9-b6714e0a702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:47.613988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:47.702271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 668/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:47.738947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:47.740531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:47.843100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb955-ff47-4375-8de5-30b6e0d1627e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:47.914696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:47.915299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:47.915576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:47.915845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:47.917460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:47.953162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:48.025328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:48.026801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d0e4b-4279-48c9-9c6b-d5bee5728c1b service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:48.044906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 672/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:52.386284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.386788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.386883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.388179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 669/1341] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:17:52.393337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fbc05-38d9-43eb-9c74-b0b75fa767d2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.394408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fbc05-38d9-43eb-9c74-b0b75fa767d2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.394745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fbc05-38d9-43eb-9c74-b0b75fa767d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.465616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-600fbc05-38d9-43eb-9c74-b0b75fa767d2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:52.466832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-600fbc05-38d9-43eb-9c74-b0b75fa767d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:52.551760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:52.563141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 673/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:52.603252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:52.604905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:52.709274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e20e3-427e-4e2b-8bd2-d1b30b0b58c9 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:52.746863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:52.782980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:52.783584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:52.783858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:52.784877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:52.786480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:52.896862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:52.898292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80addbe-71d4-4858-a9a3-d2911fa5d7ce service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:52.917469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 670/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:17:56.141099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.141615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.141615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.143034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 674/1344] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:17:56.149286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130ffc-3b84-4b1c-8e4f-8e08c7d6487f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.149719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130ffc-3b84-4b1c-8e4f-8e08c7d6487f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.149962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130ffc-3b84-4b1c-8e4f-8e08c7d6487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.216931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-30130ffc-3b84-4b1c-8e4f-8e08c7d6487f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:17:56.218122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-30130ffc-3b84-4b1c-8e4f-8e08c7d6487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:56.236050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:56.318408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 671/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:17:56.355134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:17:56.356738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:56.463101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a17d47-e15d-4d17-85dc-0e8fae8cf634 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:17:56.538841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:17:56.539462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:17:56.539739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:17:56.540008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:17:56.541742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:17:56.589201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:17:56.656135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:17:56.657606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a57fd6-804e-4890-8d8b-22a14169c05e service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:17:56.679153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 675/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:17:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:00.951883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.952357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.952448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.960228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e86b-5fd2-45a3-a4f5-2712a42ca241 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:00.960778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e86b-5fd2-45a3-a4f5-2712a42ca241 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:00.960892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e86b-5fd2-45a3-a4f5-2712a42ca241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:00.964920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 672/1347] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:00 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:01.050701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4a15e86b-5fd2-45a3-a4f5-2712a42ca241 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:01.052328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4a15e86b-5fd2-45a3-a4f5-2712a42ca241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:01.171681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.175349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 676/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:01.214040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:01.215556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:01.335690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f2a37-c61e-4a21-ad47-c9d1f84dd1a5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:01.387123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:01.415052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:01.415658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:01.415932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:01.416193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:01.417829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:01.545314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:01.546749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f137-d298-42b3-9a90-9d9a4025580b service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:01.566945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 673/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:06.469992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79961-7d3d-41fc-853a-ad8df2ca8cee None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.470436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79961-7d3d-41fc-853a-ad8df2ca8cee None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.471938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d79961-7d3d-41fc-853a-ad8df2ca8cee None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.473421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 677/1350] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:18:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:06.482237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4913e73-3a77-410a-b73e-fbb9c2c874c5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:06.482669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4913e73-3a77-410a-b73e-fbb9c2c874c5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:06.483000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4913e73-3a77-410a-b73e-fbb9c2c874c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:06.619316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f4913e73-3a77-410a-b73e-fbb9c2c874c5 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:06.620505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f4913e73-3a77-410a-b73e-fbb9c2c874c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:06.646052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:06.748523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 674/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:10.999345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8eedc-2e3e-4a7c-8b36-a0d307912e00 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:10.999868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8eedc-2e3e-4a7c-8b36-a0d307912e00 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.000119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8eedc-2e3e-4a7c-8b36-a0d307912e00 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.001661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 678/1352] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:11.007994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287e82-a3cd-4349-8b3c-2c9a16a715be None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:11.008407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287e82-a3cd-4349-8b3c-2c9a16a715be None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:11.008614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44287e82-a3cd-4349-8b3c-2c9a16a715be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:11.092242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-44287e82-a3cd-4349-8b3c-2c9a16a715be None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:11.093773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-44287e82-a3cd-4349-8b3c-2c9a16a715be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:11.175845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:11.217324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 675/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:12.382119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864ffc-ea59-4413-9a37-374f4d0f4048 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:12.382632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864ffc-ea59-4413-9a37-374f4d0f4048 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:12.382715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864ffc-ea59-4413-9a37-374f4d0f4048 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:12.384266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 679/1354] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:12.389528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807fc19-9e0f-46ec-80fd-8a1bb73ba223 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:12.389960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807fc19-9e0f-46ec-80fd-8a1bb73ba223 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:12.390083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807fc19-9e0f-46ec-80fd-8a1bb73ba223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:12.468327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c807fc19-9e0f-46ec-80fd-8a1bb73ba223 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:12.469559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c807fc19-9e0f-46ec-80fd-8a1bb73ba223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:12.576010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 676/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:18:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:16.066439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a786a-3c07-424a-84a9-10b65063c034 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.067004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a786a-3c07-424a-84a9-10b65063c034 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.067004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a786a-3c07-424a-84a9-10b65063c034 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.068342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 680/1356] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:16.074556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2b49c-0e41-46e5-be58-53f805e65b78 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.075545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2b49c-0e41-46e5-be58-53f805e65b78 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.076110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2b49c-0e41-46e5-be58-53f805e65b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.132589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.177790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c0c2b49c-0e41-46e5-be58-53f805e65b78 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:16.179515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c0c2b49c-0e41-46e5-be58-53f805e65b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:16.288991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 677/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:16.345404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.346488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.347522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.349955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 681/1358] 10.222.0.27 () {54 vars in 962 bytes} [Mon Aug 8 02:18:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:16.370916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4dea4-85de-4175-8b9e-f7140dee4b55 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.371370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4dea4-85de-4175-8b9e-f7140dee4b55 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.371751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4dea4-85de-4175-8b9e-f7140dee4b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.465382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-32c4dea4-85de-4175-8b9e-f7140dee4b55 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:16.467013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-32c4dea4-85de-4175-8b9e-f7140dee4b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:16.523839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.610611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 678/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Aug 8 02:18:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:16.618622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:16.620331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:16.776224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ad4cf-935d-413e-9c5c-632b2ea7ebe7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:16.863492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:16.911547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:16.912394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:16.912696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:16.913083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:16.914687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:17.013921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:17.015876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6068b36-86bf-4b62-99e5-14d5f1aa8794 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:17.037022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 682/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:19.302845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:19.303392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:19.303493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:19.305024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 679/1361] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:19.310679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0968955-1b03-4656-b608-f2098062d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:19.311105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0968955-1b03-4656-b608-f2098062d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:19.311276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0968955-1b03-4656-b608-f2098062d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:19.382245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e0968955-1b03-4656-b608-f2098062d3a1 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:19.383481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e0968955-1b03-4656-b608-f2098062d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:19.427683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:19.501823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 683/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:19.545147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:19.546767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:19.755265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1c0a-736f-4c38-9f9c-77eb11ede429 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:19.765245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:19.902351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:19.902996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:19.903280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:19.903731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:19.905462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:20.003497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:20.005129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979c4d9-29e6-4cd4-b849-2dd4de19f467 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:20.025104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 680/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:22.491532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52310550-b243-4be6-80bd-eb4bd5bcf697 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:22.492081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52310550-b243-4be6-80bd-eb4bd5bcf697 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:22.492081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52310550-b243-4be6-80bd-eb4bd5bcf697 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:22.493466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 684/1364] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:22.498788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124915a-f63f-4312-99ea-15caed026a90 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:22.499228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124915a-f63f-4312-99ea-15caed026a90 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:22.499402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124915a-f63f-4312-99ea-15caed026a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:22.516356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:22.573280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2124915a-f63f-4312-99ea-15caed026a90 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:22.574510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2124915a-f63f-4312-99ea-15caed026a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:22.686239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 681/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:22.734804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:22.735149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:22.735500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:22.737120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 685/1366] 10.222.0.27 () {54 vars in 962 bytes} [Mon Aug 8 02:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:22.744201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6708a-f793-498a-8523-98e3fb0e8f18 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:22.744497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6708a-f793-498a-8523-98e3fb0e8f18 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:22.744971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6708a-f793-498a-8523-98e3fb0e8f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:22.832259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-03b6708a-f793-498a-8523-98e3fb0e8f18 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:22.833700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-03b6708a-f793-498a-8523-98e3fb0e8f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:22.869895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:22.961739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 682/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Aug 8 02:18:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:22.969614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:22.972010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:23.124468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f183a7-27fc-457f-a42f-2e95eedf52e9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:23.143800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:23.243091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:23.243802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:23.244154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:23.244495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:23.246211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:23.340255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:23.342004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e02e02-9622-4741-a432-a4aa8b2ffa89 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:23.361649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 686/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:18:24.342445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea58e0-d151-4b9c-8990-d9728a32c359 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.343029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea58e0-d151-4b9c-8990-d9728a32c359 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.343029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea58e0-d151-4b9c-8990-d9728a32c359 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.344540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 683/1369] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:24.350115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855facb-bbe7-4bde-96b3-e5ccd2d6aee6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.350554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855facb-bbe7-4bde-96b3-e5ccd2d6aee6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.350733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855facb-bbe7-4bde-96b3-e5ccd2d6aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.384093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:24.420589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8855facb-bbe7-4bde-96b3-e5ccd2d6aee6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:24.421852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8855facb-bbe7-4bde-96b3-e5ccd2d6aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:24.525640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 687/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:24.878514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.878514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.878514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.882048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 684/1371] 10.222.0.27 () {54 vars in 964 bytes} [Mon Aug 8 02:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:24.884535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938e0b4-f06c-45a5-a707-991dac03f553 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:24.884874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938e0b4-f06c-45a5-a707-991dac03f553 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:24.885152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938e0b4-f06c-45a5-a707-991dac03f553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:24.967570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1938e0b4-f06c-45a5-a707-991dac03f553 None None] MFA Rules not processed for user `c925987dc66a445dae3cd8eee2f316ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:24.968843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1938e0b4-f06c-45a5-a707-991dac03f553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:25.019055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:25.049752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 688/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Aug 8 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:25.057287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:25.059094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:25.176283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf844998-6e11-4097-950f-ed11c89f39a7 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:25.222438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:25.257755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:25.258370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:25.258666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:25.258929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:25.260520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:25.374468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:25.375921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d678a78-7d60-4cfa-acae-1f935ae84c34 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:25.397084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 685/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:25.602304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063d28-49de-492c-86a4-684628aa966c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:25.602724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063d28-49de-492c-86a4-684628aa966c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:25.602981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5063d28-49de-492c-86a4-684628aa966c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:25.604219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 689/1374] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:25.610329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059ad3-9be9-4796-8231-f5f659ff7cd5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:25.610768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059ad3-9be9-4796-8231-f5f659ff7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:25.611898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11059ad3-9be9-4796-8231-f5f659ff7cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:25.684961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-11059ad3-9be9-4796-8231-f5f659ff7cd5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:25.686258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-11059ad3-9be9-4796-8231-f5f659ff7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:25.711616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:25.812993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 686/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:25.851274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5063d28-49de-492c-86a4-684628aa966c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:25.853011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f5063d28-49de-492c-86a4-684628aa966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:26.000196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5063d28-49de-492c-86a4-684628aa966c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:26.052240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:26.114935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:26.115544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:26.115825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:26.116096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:26.117742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:26.209353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:26.210952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99d3c5-1356-4a19-a0e5-072e2b2ce573 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:26.229251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 690/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:28.419608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:28.420495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:28.420805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:28.422160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 687/1377] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:28.428369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abe2c-bc59-4a6a-8b17-986e240bb2a8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:28.428830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abe2c-bc59-4a6a-8b17-986e240bb2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:28.429071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abe2c-bc59-4a6a-8b17-986e240bb2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:28.494987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-dc8abe2c-bc59-4a6a-8b17-986e240bb2a8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:28.496230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dc8abe2c-bc59-4a6a-8b17-986e240bb2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:28.506333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:28.628193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 691/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:28.676180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:28.677794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:28.828716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7a1e3-887f-43e8-833d-120d469bf29b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:28.832718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:28.951225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:28.952042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:28.952481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:28.952793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:28.954443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:29.079458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:29.081756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ab184-767f-4e09-b8d5-9c80a0c1ab55 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:29.102920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 688/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:31.391604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:31.392156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:31.392156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:31.394538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 692/1380] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:31.399729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df9de-bcab-40f8-b77a-85e49be1646b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:31.403612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df9de-bcab-40f8-b77a-85e49be1646b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:31.403612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df9de-bcab-40f8-b77a-85e49be1646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:31.413294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:31.475137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1d8df9de-bcab-40f8-b77a-85e49be1646b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:31.476462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1d8df9de-bcab-40f8-b77a-85e49be1646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:31.599466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 689/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:31.631527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:31.633063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:31.779037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:31.790482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b8ad-c6c8-4fdc-b680-35c6ed6fc4aa None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:31.909938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:31.910460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:31.910750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:31.911022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:31.912652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:32.016758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:32.018633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a782712-899b-448f-9277-18f8de7b90c2 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:32.030232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:32.039425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 693/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:34.277221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:34.277767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:34.277767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:34.279134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 690/1383] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:34.284903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c58c7-955e-4464-91f2-78dfa530d27d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:34.285339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c58c7-955e-4464-91f2-78dfa530d27d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:34.285483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c58c7-955e-4464-91f2-78dfa530d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:34.353555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ac3c58c7-955e-4464-91f2-78dfa530d27d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:34.354774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ac3c58c7-955e-4464-91f2-78dfa530d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:34.454519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 694/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:34.498300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:34.500369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:34.596350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:34.669025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de97d94-89d4-47ef-a929-907ca10af35b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:34.811850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:34.812322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:34.812599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:34.812900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:34.815248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:34.865404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:34.905871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:34.907483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd159e72-fa5d-4eee-be1e-82f1ea2f39d0 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:34.927660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 691/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:36.950280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:36.950796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:36.950796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:36.952150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 695/1386] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:36.957715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd47bb-90ca-4eef-af19-6e34b4b302de None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:36.958266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd47bb-90ca-4eef-af19-6e34b4b302de None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:36.958266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd47bb-90ca-4eef-af19-6e34b4b302de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:37.041060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-48fd47bb-90ca-4eef-af19-6e34b4b302de None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:37.042275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-48fd47bb-90ca-4eef-af19-6e34b4b302de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:37.156405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 692/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:37.206517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:37.208573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:37.289262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:37.358677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483d34-77cd-425d-8df9-f4ae1b1cdff2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:37.473710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:37.474222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:37.474507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:37.474838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:37.476368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:37.514599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:37.564994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:37.566759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ca543-2abe-4d46-813d-135fe686ae27 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:37.589325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 696/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:40.132568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.133192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.133192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.134479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 693/1389] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:40.139600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148e414-b5b2-4058-9bd5-92814c72b067 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.140016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148e414-b5b2-4058-9bd5-92814c72b067 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.140149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148e414-b5b2-4058-9bd5-92814c72b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.208083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e148e414-b5b2-4058-9bd5-92814c72b067 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:40.209320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e148e414-b5b2-4058-9bd5-92814c72b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:40.308071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 697/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:40.352726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:40.354370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:40.387477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:40.503130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6682c-7db3-4b16-b6c1-1337e5be5b84 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:40.610068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:40.621960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:40.622573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.622851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.623121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.624756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:40.837487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:40.839288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273e3ec-4150-41b7-ad69-54f4a87242fa service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:40.858138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 694/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:40.942592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eccd64-bd44-4dc6-8637-c0946972a2f8 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.942878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eccd64-bd44-4dc6-8637-c0946972a2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.943157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eccd64-bd44-4dc6-8637-c0946972a2f8 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:40.944546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 698/1392] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:40.950042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29b39-b826-4fe5-830a-24ece80ac3ef None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:40.950327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29b39-b826-4fe5-830a-24ece80ac3ef None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:40.950603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e29b39-b826-4fe5-830a-24ece80ac3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:41.016097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-73e29b39-b826-4fe5-830a-24ece80ac3ef None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:41.017449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-73e29b39-b826-4fe5-830a-24ece80ac3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:41.079163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:41.123186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 695/1393] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:41.183694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920982ed-1634-474e-85f8-d213be83705c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:41.184283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920982ed-1634-474e-85f8-d213be83705c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:41.184283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920982ed-1634-474e-85f8-d213be83705c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:41.185781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 699/1394] 10.222.0.27 () {54 vars in 966 bytes} [Mon Aug 8 02:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:41.191815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db0ce-b27e-409c-a6e1-09d912b35ff9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:41.192339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db0ce-b27e-409c-a6e1-09d912b35ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:41.192476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233db0ce-b27e-409c-a6e1-09d912b35ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:41.282697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-233db0ce-b27e-409c-a6e1-09d912b35ff9 None None] MFA Rules not processed for user `144f1e9a377c4dd098112690cfdf24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:41.283940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-233db0ce-b27e-409c-a6e1-09d912b35ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:41.371696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 696/1395] 10.222.0.27 () {58 vars in 1063 bytes} [Mon Aug 8 02:18:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:41.379727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920982ed-1634-474e-85f8-d213be83705c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:41.381197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-920982ed-1634-474e-85f8-d213be83705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:41.395614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:41.492438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920982ed-1634-474e-85f8-d213be83705c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:41.567739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:41.568355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:41.568638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:41.568948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:41.570545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:41.639905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:41.680375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:41.682264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4d98d-149a-4ea9-8c51-0e9073f3fc58 service glance] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:41.701363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 700/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:42.575867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38c11-111f-4006-9d6d-c33614acd032 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:42.576923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38c11-111f-4006-9d6d-c33614acd032 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:42.576923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38c11-111f-4006-9d6d-c33614acd032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:42.644428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f3f38c11-111f-4006-9d6d-c33614acd032 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:42.645658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f3f38c11-111f-4006-9d6d-c33614acd032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:42.654635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:42.732515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 697/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:18:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:42.778770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:42.779238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:42.779332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:42.780754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 701/1398] 10.222.0.27 () {54 vars in 943 bytes} [Mon Aug 8 02:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:42.787385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcd674-b099-47c3-bf1f-4723e67da1b9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:42.787666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcd674-b099-47c3-bf1f-4723e67da1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:42.787956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fcd674-b099-47c3-bf1f-4723e67da1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:42.865177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-68fcd674-b099-47c3-bf1f-4723e67da1b9 None None] MFA Rules not processed for user `171d2d63e59b41a783cdbc61db788122`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:42.866286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-68fcd674-b099-47c3-bf1f-4723e67da1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:42.949712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:42.973255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 698/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Mon Aug 8 02:18:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:42.982749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:42.984309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:43.055009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a11de-c54d-43f4-b935-483780f96cda None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:43.055295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a11de-c54d-43f4-b935-483780f96cda None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:43.055571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a11de-c54d-43f4-b935-483780f96cda None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:43.056955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 699/1400] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:43.062027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:43.062310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:43.062583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:43.122850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714854-45ca-45da-b002-0c990c11fad8 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:43.128851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:43.138142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:43.163020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:43.231552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:43.232166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:43.232357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 700/1401] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:18:43.232477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:43.232724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:43.234488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:43.280202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:43.282073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:43.330372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:43.332848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c92295-0f21-4277-9bb0-efd97c6a8a88 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:43.340985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 702/1402] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:18:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:18:43.356351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:43.425768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536df41-3371-4928-a448-776ab70a1b01 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:43.531466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:43.532069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:43.532343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:43.532610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:43.535867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:43.614750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:43.629275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:43.630855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f364f91-f8b5-4487-975e-c640634d76a3 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:43.649685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 701/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:18:45.863572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:45.864109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:45.864205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:45.865654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 703/1404] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:45.870825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d618a-be07-494d-9db2-9ee10e0e4546 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:45.871279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d618a-be07-494d-9db2-9ee10e0e4546 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:45.871403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d618a-be07-494d-9db2-9ee10e0e4546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:45.945526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e48d618a-be07-494d-9db2-9ee10e0e4546 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:45.946755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e48d618a-be07-494d-9db2-9ee10e0e4546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:46.057000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 702/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:46.104528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:46.106341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:46.187603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:46.272413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c35b-a098-4041-8886-07b3003d83fb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:46.414746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:46.415987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:46.416283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:46.416560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:46.418305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:46.454521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:46.532646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:46.534361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd224e-b887-4f9c-ad7e-386336c42246 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:46.554106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 704/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:48.831689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:48.832375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:48.832375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:48.833839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 703/1407] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:18:48.839382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9117a-97d9-4305-8729-61745dfc3836 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:48.839382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9117a-97d9-4305-8729-61745dfc3836 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:48.839952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9117a-97d9-4305-8729-61745dfc3836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:48.920393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ece9117a-97d9-4305-8729-61745dfc3836 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:18:48.921300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ece9117a-97d9-4305-8729-61745dfc3836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:49.035984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 705/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:18:49.082953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:18:49.084453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:49.127343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:49.251057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1baff8-ad3a-4802-ac27-4e9885da3b99 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:18:49.373969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:49.379834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:18:49.380429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:18:49.380730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:18:49.387544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:18:49.387544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:18:49.489912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:18:49.491503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dca65-0229-4bda-9786-6d1b6239c318 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:18:49.511580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 704/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:18:59.032457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:18:59.509514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=47 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:07.157984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759729-8eea-4243-9262-337e39e1de2d None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:07.158547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759729-8eea-4243-9262-337e39e1de2d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:07.158547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e759729-8eea-4243-9262-337e39e1de2d None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:07.159877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 706/1410] 10.222.0.27 () {54 vars in 918 bytes} [Mon Aug 8 02:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:07.164985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdc16-eb5f-4b37-a40a-ed0bddebbe11 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:07.165389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdc16-eb5f-4b37-a40a-ed0bddebbe11 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:07.165529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdc16-eb5f-4b37-a40a-ed0bddebbe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:07.241081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-239fdc16-eb5f-4b37-a40a-ed0bddebbe11 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:07.242288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-239fdc16-eb5f-4b37-a40a-ed0bddebbe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:07.348365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 705/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Aug 8 02:19:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:07.363021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50b0a-aa5e-4e2c-89c3-7c08ec099308 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:07.363305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50b0a-aa5e-4e2c-89c3-7c08ec099308 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:07.363576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc50b0a-aa5e-4e2c-89c3-7c08ec099308 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:07.364943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 707/1412] 10.222.0.27 () {54 vars in 918 bytes} [Mon Aug 8 02:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:07.369746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47f363-4bdb-427d-9d9f-56f5977dd9c0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:07.370037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47f363-4bdb-427d-9d9f-56f5977dd9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:07.370314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47f363-4bdb-427d-9d9f-56f5977dd9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:07.442784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:07.445449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3b47f363-4bdb-427d-9d9f-56f5977dd9c0 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:07.446653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3b47f363-4bdb-427d-9d9f-56f5977dd9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:07.560445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 706/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Aug 8 02:19:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:10.048010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:10.048570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:10.048570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:10.050683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:10.055776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d504a-be92-435f-951f-dd5f0eb80c4e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:10.056214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d504a-be92-435f-951f-dd5f0eb80c4e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:10.056321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d504a-be92-435f-951f-dd5f0eb80c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:10.121078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-689d504a-be92-435f-951f-dd5f0eb80c4e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:10.122278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-689d504a-be92-435f-951f-dd5f0eb80c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.176140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:10.218052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:10.260743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:10.262375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.407534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b826a8-0bda-41e3-964c-1eb980186b81 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:10.478935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:10.518672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:10.519414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:10.519697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:10.519972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:10.521686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:10.609641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:10.611244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0fe8c-f7ae-4e4c-9883-a5bfc6df2509 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:10.630168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 709/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:11.526031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:11.526618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:11.526618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:11.527932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:11.532841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2934f-52d9-4d68-a84e-7675a112e46e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:11.533226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2934f-52d9-4d68-a84e-7675a112e46e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:11.533415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2934f-52d9-4d68-a84e-7675a112e46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:11.610663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7ee2934f-52d9-4d68-a84e-7675a112e46e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:11.611918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7ee2934f-52d9-4d68-a84e-7675a112e46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:11.638132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:11.707642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:11.860299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:11.862399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:12.006890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367200c-3881-4a4f-95e9-6fa442e9b112 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:12.046992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:12.118322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:12.118847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:12.119141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:12.119404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:12.121003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:12.207532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:12.209139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c047e4-f17b-4c24-b630-2bbb1da71350 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:12.227011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 709/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:13.124307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:13.124867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:13.124961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:13.126255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:13.131499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e131-ec25-4457-803a-eb6dad8c5f85 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:13.131882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e131-ec25-4457-803a-eb6dad8c5f85 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:13.139633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9e131-ec25-4457-803a-eb6dad8c5f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:13.187161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:13.208613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-72d9e131-ec25-4457-803a-eb6dad8c5f85 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:13.209824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-72d9e131-ec25-4457-803a-eb6dad8c5f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:13.306278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:13.459566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:13.461190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:13.602276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7380-06ed-46e2-a13a-bc842b7003f2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:13.614174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:13.713395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:13.714002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:13.714281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:13.714552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:13.716142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:13.803289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:13.804915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12476ac-4681-4707-a88b-1d4283187771 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:13.823963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 712/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:14.682254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:14.683084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:14.683084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:14.684476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:14.689708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80530-8610-4973-8746-3b45c51cd9c4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:14.690659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80530-8610-4973-8746-3b45c51cd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:14.690947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80530-8610-4973-8746-3b45c51cd9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:14.711422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:14.769378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e7b80530-8610-4973-8746-3b45c51cd9c4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:14.770583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e7b80530-8610-4973-8746-3b45c51cd9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:14.988261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:15.149061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:15.150558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:15.260629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:15.296199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37bebb-85a5-4506-b695-8e9d61791049 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:15.406937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:15.407544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:15.407820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:15.408091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:15.410243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:15.485610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:15.499179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:15.500797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aef7a-b24e-47a3-ad38-de7c1ffd9d7a service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:15.520018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 712/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:16.405098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:16.405630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:16.405630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:16.407013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:16.412197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d84e-b156-4cd0-a4ba-42f19fdc6e76 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:16.412624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d84e-b156-4cd0-a4ba-42f19fdc6e76 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:16.412767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380d84e-b156-4cd0-a4ba-42f19fdc6e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:16.489205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f380d84e-b156-4cd0-a4ba-42f19fdc6e76 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:16.490421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f380d84e-b156-4cd0-a4ba-42f19fdc6e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:16.598108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:16.753370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:16.754917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:16.839916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:16.901655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a47649-b601-4fe5-8703-a13b7fe0db2c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:17.012690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:17.013305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:17.013582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:17.013851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:17.015438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:17.062737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:17.103886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:17.105439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5360ec-717b-4f21-b00a-77e8cff2f93f service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:17.123536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 715/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:18.013272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:18.013794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:18.013794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:18.015158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:18.020745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4f68-94cb-49d8-9445-837d16bedc09 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:18.021156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4f68-94cb-49d8-9445-837d16bedc09 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:18.021309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4f68-94cb-49d8-9445-837d16bedc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:18.100247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-41ac4f68-94cb-49d8-9445-837d16bedc09 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:18.101471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-41ac4f68-94cb-49d8-9445-837d16bedc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:18.196194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:18.350807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:18.352333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:18.401898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:18.498084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070ed7-5eb8-451d-bc91-44f4aa83bfb3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:18.610802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:18.611412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:18.612231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:18.612609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:18.614242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:18.621127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:18.702934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:18.704526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcdbad-ffb7-4e5a-ac5c-c820f383a454 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:18.722137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 715/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:19.601276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:19.601862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:19.601862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:19.603157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:19.608262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e831-54fd-405f-8ec1-064fde15e01c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:19.608770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e831-54fd-405f-8ec1-064fde15e01c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:19.608884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e831-54fd-405f-8ec1-064fde15e01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:19.685668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-4bf3e831-54fd-405f-8ec1-064fde15e01c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:19.686909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4bf3e831-54fd-405f-8ec1-064fde15e01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:19.756116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:19.786056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:19:19.938610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:19.940586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:19.958400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:20.082581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555a2f7-a597-4142-8e04-d28437ee5abe None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:20.168793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:20.194317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:20.195032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:20.195521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:20.196091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:20.197733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:20.285611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:20.287190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d1b64-0f60-465d-a17f-6991afef2e83 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:20.304793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 718/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:21.170283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:21.170859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:21.170859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:21.172214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:21.177305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307f8-476c-4c1c-ba51-bea60ca49a04 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:21.177793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307f8-476c-4c1c-ba51-bea60ca49a04 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:21.177925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53307f8-476c-4c1c-ba51-bea60ca49a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:21.255849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e53307f8-476c-4c1c-ba51-bea60ca49a04 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:21.257127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e53307f8-476c-4c1c-ba51-bea60ca49a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:21.298611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:21.353151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:21.505211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:21.506728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:21.651771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff701a5-eb1e-47bd-af2a-bea0ffd069f7 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:21.711211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:21.768767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639fd49-7710-4be5-8733-11947906308d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:21.769315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639fd49-7710-4be5-8733-11947906308d service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:21.769593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639fd49-7710-4be5-8733-11947906308d service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:21.769877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639fd49-7710-4be5-8733-11947906308d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:21.771449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0639fd49-7710-4be5-8733-11947906308d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:21.857722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639fd49-7710-4be5-8733-11947906308d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:21.859307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639fd49-7710-4be5-8733-11947906308d service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:21.878131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 718/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:22.738712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:22.739298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:22.739298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:22.740627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:22.745516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4661a46-864c-4101-888d-6c5ca8fe8e60 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:22.745955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4661a46-864c-4101-888d-6c5ca8fe8e60 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:22.746076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4661a46-864c-4101-888d-6c5ca8fe8e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:22.822162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b4661a46-864c-4101-888d-6c5ca8fe8e60 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:22.823358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b4661a46-864c-4101-888d-6c5ca8fe8e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:22.825389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:22.918959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:23.071944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:23.073563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:23.219314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbac97-a78e-4976-96fc-e7f362bcde4e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:23.245814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:23.331350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:23.331954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:23.332231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:23.332498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:23.334122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:23.422212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:23.423835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a8e0e-6f53-402d-85e4-d345ac4b0f6e service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:23.442032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 721/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:25.359080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:25.359614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:25.359614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:25.360932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:25.366695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f920-6741-4717-a491-c636b87ff500 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:25.367129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f920-6741-4717-a491-c636b87ff500 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:25.367233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f920-6741-4717-a491-c636b87ff500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:25.407355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:25.434706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b7c0f920-6741-4717-a491-c636b87ff500 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:25.435909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b7c0f920-6741-4717-a491-c636b87ff500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:25.531441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:25.570861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:25.572385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:25.675890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43740484-bcf8-448a-820f-1a916ee71901 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:25.707589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:25.748687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:25.749295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:25.749574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:25.749851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:25.751432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:25.858170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:25.859598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ef1e0b-f39e-4b48-9e26-de239d41097a service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:25.878307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 721/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:28.031695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:28.032225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:28.032225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:28.034109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:28.039611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73f36-1a7c-458d-a828-5f20c158e993 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:28.040084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73f36-1a7c-458d-a828-5f20c158e993 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:28.040193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d73f36-1a7c-458d-a828-5f20c158e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:28.107289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f3d73f36-1a7c-458d-a828-5f20c158e993 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:28.108488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f3d73f36-1a7c-458d-a828-5f20c158e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:28.121118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:28.204614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:19:28.244496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:28.246052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:28.348560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a3fa0-0082-4d4f-87de-09d4af68a99c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:28.360630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:28.421632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:28.422228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:28.422505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:28.422771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:28.424916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:28.531408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:28.532798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d50277-3076-48a2-b278-07cf7d83224a service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:28.550856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 724/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:30.949754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:30.950304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:30.950304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:30.951690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:30.957612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cca0c-6b45-418f-abf1-94ef40736ba3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:30.958028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cca0c-6b45-418f-abf1-94ef40736ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:30.958159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cca0c-6b45-418f-abf1-94ef40736ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:31.006237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:31.024378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-999cca0c-6b45-418f-abf1-94ef40736ba3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:31.025641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-999cca0c-6b45-418f-abf1-94ef40736ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:31.122697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:31.163062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:31.164660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:31.266563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d8d27-a1d1-4f3e-a5bb-85fbc719dd11 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:31.306458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:31.340913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:31.341523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:31.341806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:31.342081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:31.343692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:31.452225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:31.453681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89079f25-7f7a-4237-aaab-ccda8678c66e service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:31.471599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 724/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:33.616370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:33.616922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:33.616922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:33.618273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:33.623418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94cea8-a88c-4f55-a80c-de528d1ccb38 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:33.623815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94cea8-a88c-4f55-a80c-de528d1ccb38 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:33.623959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94cea8-a88c-4f55-a80c-de528d1ccb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:33.691045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5f94cea8-a88c-4f55-a80c-de528d1ccb38 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:33.692238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5f94cea8-a88c-4f55-a80c-de528d1ccb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:33.712586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:33.787423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:33.827117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:33.828706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:33.931162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb15ab-b5d4-4896-961c-07ff0e9dab4d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:33.951417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:34.005443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:34.006049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:34.006329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:34.006603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:34.008205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:34.115317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:34.116769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78ee62-9053-46f4-aacf-f033dc9ed6ac service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:34.134584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 727/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:36.212717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:36.213308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:36.213308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:36.214619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:36.219946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b156c-109b-4bb4-8944-02f7ad5055c5 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:36.220342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b156c-109b-4bb4-8944-02f7ad5055c5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:36.221093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b156c-109b-4bb4-8944-02f7ad5055c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:36.283795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:36.286492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-203b156c-109b-4bb4-8944-02f7ad5055c5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:36.287698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-203b156c-109b-4bb4-8944-02f7ad5055c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:36.382757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:36.420824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:36.422363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:36.526062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2cab8-5f33-4221-9ca6-c74983387cb0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:36.573939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:36.599191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:36.599831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:36.600108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:36.600374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:36.601998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:36.708380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:36.709858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939ebf9-bb10-43ea-b6a3-38294c2070d1 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:36.727707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 727/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:38.788876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:38.789453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:38.789453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:38.790771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:38.796069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce136c-2058-43d9-9f3d-80fd438f66a5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:38.796505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce136c-2058-43d9-9f3d-80fd438f66a5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:38.796641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce136c-2058-43d9-9f3d-80fd438f66a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:38.862707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-99ce136c-2058-43d9-9f3d-80fd438f66a5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:38.863840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-99ce136c-2058-43d9-9f3d-80fd438f66a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:38.893823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:38.961233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:38.999759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:39.001376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:39.102752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb778ad-1e9a-4de9-9b06-73f29f32d330 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:39.131691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:39.176709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:39.177504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:39.178176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:39.178452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:39.180059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:39.289280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:39.291787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9128-a2f6-4b8a-8cf1-d718f03620e4 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:39.310365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 730/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:41.369890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e538a8-1346-4807-96df-26d303f88943 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:41.370424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e538a8-1346-4807-96df-26d303f88943 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:41.370424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e538a8-1346-4807-96df-26d303f88943 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:41.371698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:41.377789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d45e-8e4e-465f-93d7-82aa0fa15c01 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:41.378563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d45e-8e4e-465f-93d7-82aa0fa15c01 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:41.378842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35d45e-8e4e-465f-93d7-82aa0fa15c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:41.443523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2f35d45e-8e4e-465f-93d7-82aa0fa15c01 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:41.444747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2f35d45e-8e4e-465f-93d7-82aa0fa15c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:41.452766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:41.540534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:41.579423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e538a8-1346-4807-96df-26d303f88943 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:41.581338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-76e538a8-1346-4807-96df-26d303f88943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:41.685190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e538a8-1346-4807-96df-26d303f88943 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:41.739079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:41.758616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:41.759244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:41.759520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:41.759786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:41.761420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:41.870686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:41.872349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c837ef-882e-4297-a12f-0810a795aefd service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:41.890232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 730/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:43.942221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:43.942752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:43.942752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:43.944041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:43.949379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da6a8-d002-42d2-844f-263bdab46d41 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:43.949758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da6a8-d002-42d2-844f-263bdab46d41 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:43.949939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757da6a8-d002-42d2-844f-263bdab46d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:44.015730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-757da6a8-d002-42d2-844f-263bdab46d41 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:44.016995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-757da6a8-d002-42d2-844f-263bdab46d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:44.054866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:44.113998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:44.151973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:44.153580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:44.261751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf7be-6bb6-47e6-b6e2-e186529c3492 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:44.299643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:44.335242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:44.335857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:44.336138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:44.336411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:44.338050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:44.445883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:44.447303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0452fcf-ef58-4754-809a-2ba376fdcb77 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:44.464958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 733/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:46.487656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.488215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.488313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.489703 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:19:46.494934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26705f-7347-4b7c-8b9b-9fa66738ca45 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.495373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26705f-7347-4b7c-8b9b-9fa66738ca45 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.495475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26705f-7347-4b7c-8b9b-9fa66738ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.561881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-cb26705f-7347-4b7c-8b9b-9fa66738ca45 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:46.563083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cb26705f-7347-4b7c-8b9b-9fa66738ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:46.581054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:46.659311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:46.695885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:46.697457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:46.801802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a789b6-fb6b-4d00-9911-5b761010ff18 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:46.871812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:46.880179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:46.881014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:46.881363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:46.881682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:46.883385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:46.999680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:47.001186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd033345-f52a-48b0-8144-f3ca39a2bd6f service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:47.021823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 733/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:49.104869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:49.105397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:49.105397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:49.106767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:19:49.112537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0041d-c276-4bb2-b651-0083f6c5971b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:49.112954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0041d-c276-4bb2-b651-0083f6c5971b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:49.113127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0041d-c276-4bb2-b651-0083f6c5971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:49.179793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fec0041d-c276-4bb2-b651-0083f6c5971b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:19:49.181020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fec0041d-c276-4bb2-b651-0083f6c5971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:49.229691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:49.278560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:19:49.316185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:19:49.317831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:49.419344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860f5b-88f6-4a21-bd34-3fd72ebed7db None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:19:49.467151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:49.493543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:19:49.494151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:19:49.494431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:19:49.494700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:19:49.497058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:19:49.620222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:19:49.622362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54938e94-66ec-4379-b1f9-420d86cb2bd7 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:19:49.668154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 736/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:19:59.276787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=22 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:19:59.639364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=66 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:09.379897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74513-4b9d-43ea-aea4-0d10c7962c74 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.380435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74513-4b9d-43ea-aea4-0d10c7962c74 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.380435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74513-4b9d-43ea-aea4-0d10c7962c74 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.381764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 735/1471] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:09.386985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d5d2-1ccc-4991-820d-2950267cf234 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.387411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d5d2-1ccc-4991-820d-2950267cf234 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.387536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6d5d2-1ccc-4991-820d-2950267cf234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.452219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-96b6d5d2-1ccc-4991-820d-2950267cf234 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:09.453459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-96b6d5d2-1ccc-4991-820d-2950267cf234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.548251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 737/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:09.634310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.634534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.634806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.636125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 736/1473] 10.222.0.27 () {54 vars in 966 bytes} [Mon Aug 8 02:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:09.640897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f321d-1c37-410a-8bf3-e5f6eb636981 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.641190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f321d-1c37-410a-8bf3-e5f6eb636981 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.641517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f321d-1c37-410a-8bf3-e5f6eb636981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.716355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-df0f321d-1c37-410a-8bf3-e5f6eb636981 None None] MFA Rules not processed for user `144f1e9a377c4dd098112690cfdf24bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:09.717588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-df0f321d-1c37-410a-8bf3-e5f6eb636981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.738225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:09.792428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 738/1474] 10.222.0.27 () {58 vars in 1063 bytes} [Mon Aug 8 02:20:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:09.800538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:09.802136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:09.906148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194775c-d9d2-40aa-bff5-e97b7d77b007 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:09.979118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:09.979722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:09.980002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:09.980279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:09.981922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:10.035827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:10.090575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:10.091987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1663a4-b94c-4703-8f50-1540c0935afd service glance] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:10.111815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 737/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:12.207254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:12.207799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:12.207799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:12.209189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 739/1476] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:12.214396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c2b6b-6b1e-4a2b-a7e8-8e88523ec8fa None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:12.214814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c2b6b-6b1e-4a2b-a7e8-8e88523ec8fa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:12.214930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c2b6b-6b1e-4a2b-a7e8-8e88523ec8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:12.283050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-284c2b6b-6b1e-4a2b-a7e8-8e88523ec8fa None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:12.284357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-284c2b6b-6b1e-4a2b-a7e8-8e88523ec8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:12.382450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 738/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:12.405987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:12.407536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:12.532439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdeb244-d7d4-44fa-bb94-8e65e63d9308 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:12.580520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:12.625311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:12.626487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:12.626761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:12.627030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:12.627667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:12.629414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:20:12.629414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:20:12.629414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:20:12.630551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9e5a4-e906-47bb-bce0-c422850679ed admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:12.648614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 740/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Aug 8 02:20:12 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:20:14.656543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:14.657608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:14.657827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:14.659235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 739/1479] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:14.664470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fd661-96fd-47e8-9f08-52629e349339 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:14.664946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fd661-96fd-47e8-9f08-52629e349339 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:14.665056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fd661-96fd-47e8-9f08-52629e349339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:14.731292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6b2fd661-96fd-47e8-9f08-52629e349339 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:14.732529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6b2fd661-96fd-47e8-9f08-52629e349339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:14.827215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 741/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:14.871092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:14.872599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:14.882952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:15.017022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978cc79-e241-4959-bb9f-58af002ab110 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:15.095030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:15.128894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:15.129430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:15.129710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:15.129981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:15.131584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:15.220176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:15.221976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bd566-0455-4476-bc6f-927651e88564 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:15.239843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 740/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:20:17.473218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:17.473753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:17.473753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:17.475024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 742/1482] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:17.480129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd343a-25ca-4cef-b4c8-e093a2d12558 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:17.480577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd343a-25ca-4cef-b4c8-e093a2d12558 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:17.480716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd343a-25ca-4cef-b4c8-e093a2d12558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:17.546734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5ebd343a-25ca-4cef-b4c8-e093a2d12558 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:17.548559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5ebd343a-25ca-4cef-b4c8-e093a2d12558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:17.595823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:17.647477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 741/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:20:17.685778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:17.687325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:17.710550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:17.790293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b7c6c-210f-4947-a2ba-7cc0e9451781 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:17.865073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:17.865683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:17.865964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:17.866243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:17.867842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:17.943429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:17.978630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:17.980057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef69eec-f4df-4194-bfb3-fa3882f9afb6 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:17.999243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 743/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:20.086327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:20.086898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:20.086898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:20.088200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 742/1485] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:20:20.093345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253390-ebbc-423d-9d69-f0d8ff5b01a0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:20.093804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253390-ebbc-423d-9d69-f0d8ff5b01a0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:20.093921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69253390-ebbc-423d-9d69-f0d8ff5b01a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:20.160542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-69253390-ebbc-423d-9d69-f0d8ff5b01a0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:20.161784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-69253390-ebbc-423d-9d69-f0d8ff5b01a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:20.257089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 744/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:20.298219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:20.299679 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:20.418606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3fd6-4d04-4886-b5ed-44dfbc5c4f32 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:20.482991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:20.484850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:20.485582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:20.485860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:20.486131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:20.487714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:20.596380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:20.597867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284865-4499-43eb-9ccc-8f60f10afdf0 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:20.615834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 743/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:22.774482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:22.775060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:22.775060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:22.776422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 745/1488] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:22.782276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefa99f-511d-4e1e-bff8-90feda6a6cdb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:22.782817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefa99f-511d-4e1e-bff8-90feda6a6cdb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:22.782817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefa99f-511d-4e1e-bff8-90feda6a6cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:22.849856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fdefa99f-511d-4e1e-bff8-90feda6a6cdb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:22.850977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fdefa99f-511d-4e1e-bff8-90feda6a6cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:22.904108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:22.949253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 744/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:22.991176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:22.992747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:23.029602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:23.097450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c323b9-0115-4cbf-8be6-0c6357f544d1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:23.169747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:23.170275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:23.170555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:23.170949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:23.172755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:23.250716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:23.281290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:23.283863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef4424-305c-4927-83ca-756fcbb25c29 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:23.301988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 746/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:25.666372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:25.666925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:25.666925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:25.668245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 745/1491] 10.222.0.27 () {54 vars in 927 bytes} [Mon Aug 8 02:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:20:25.673499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217ef1d-9fd3-4181-b137-aad75be1113e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:25.674001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217ef1d-9fd3-4181-b137-aad75be1113e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:25.674109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217ef1d-9fd3-4181-b137-aad75be1113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:25.738883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9217ef1d-9fd3-4181-b137-aad75be1113e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:20:25.740092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9217ef1d-9fd3-4181-b137-aad75be1113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:25.837167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 747/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Aug 8 02:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:20:25.876278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:20:25.877723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:25.981366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52924a-855e-4e17-b842-ba73d4edb20f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:20:26.054387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:20:26.054989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:20:26.055267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:20:26.055535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:20:26.057145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:20:26.066844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:26.166555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:20:26.167910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c96b76-680c-493c-b315-0e196b5b0aad service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:20:26.186653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 746/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:20:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:20:35.835233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=82 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:20:36.184358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=50 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:44.719444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:44.720028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:44.720028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:44.794038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:44.795232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:44.889293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 748/1494] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:21:44.897912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:44.898403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:44.898625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:44.975248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:44.976434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:45.076826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 747/1495] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:21:45.081759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.083046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:45.094492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e97d-3c91-4cfa-b872-6ac3be5ed089 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:45.155201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:45.189640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:45.190248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:45.190529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:45.190808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:45.191448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:21:45.191751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:45.192804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:45.192804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:21:45.192804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:21:45.193684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:45.204607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 749/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:21:45.213097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.222564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6ceaa-66c0-4b74-8ea0-7c9322f3aee2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:45.282155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:45.315583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:45.316186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:45.316459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:45.316753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:45.317384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:45.319113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:45.319113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:21:45.319113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:21:45.320299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:45.432500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 748/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:21:45.437124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.439116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:45.473862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e051e-3bcb-4abf-876f-e2457c435204 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:45.566503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:45.567103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:45.567381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:45.567646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:45.568269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:45.569305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:45.569305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:21:45.569305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:21:45.570501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:45.683735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 750/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:21:45 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:21:45.688027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.689583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:45.751472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fdb6f-32d5-4745-87e6-837df86899a4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:45.833258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:45.843884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:45.846351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:45.846351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:45.846351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:45.846351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:45.846838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:45.846838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:21:45.846838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:21:45.847595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:45.859777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 749/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:21:45.863818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.865442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:45.876995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc464ae-d350-4703-a788-88f54f6f2687 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:45.902020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:45.972375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:45.972941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:45.973218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:45.973486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:45.974123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:45.975211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:45.975211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:21:45.975211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:21:45.976103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80cec8-c06c-405a-948e-01d5895999f5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:45.988453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 751/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:21:45.992754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:45.994302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:46.029632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8aa5a-ed19-480c-b00c-f6033b5300e6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:46.122579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:46.123228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.123509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.123784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] PATH_INFO: `/v3/projects/9fcc3b45fa7e4f4f90232766c498c5fb/users/65d8037802f241878a38c92d261e3616/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.146612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcc3b45fa7e4f4f90232766c498c5fb, user_id=65d8037802f241878a38c92d261e3616, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:46.150212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:46.150212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:21:46.150212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:21:46.150212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75945c74-38f2-4743-a3b8-af4355623256 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:46.254576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 750/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:21:45 2022] PUT /identity/v3/projects/9fcc3b45fa7e4f4f90232766c498c5fb/users/65d8037802f241878a38c92d261e3616/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:21:46.267227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79bff-6bed-44c4-a1fe-e9b3c56d7027 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.267511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79bff-6bed-44c4-a1fe-e9b3c56d7027 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.267781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79bff-6bed-44c4-a1fe-e9b3c56d7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.326554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-34b79bff-6bed-44c4-a1fe-e9b3c56d7027 None None] MFA Rules not processed for user `65d8037802f241878a38c92d261e3616`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:46.328172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-34b79bff-6bed-44c4-a1fe-e9b3c56d7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:46.405489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 752/1502] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:21:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:21:46.419159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.419444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.419721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.421107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 751/1503] 10.222.0.27 () {54 vars in 964 bytes} [Mon Aug 8 02:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:21:46.427552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab708-4528-4f93-ad62-bb8f8cc8bad0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.427840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab708-4528-4f93-ad62-bb8f8cc8bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.428110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab708-4528-4f93-ad62-bb8f8cc8bad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.459518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:46.505048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-67dab708-4528-4f93-ad62-bb8f8cc8bad0 None None] MFA Rules not processed for user `c925987dc66a445dae3cd8eee2f316ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:46.506184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-67dab708-4528-4f93-ad62-bb8f8cc8bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:46.580205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 753/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Aug 8 02:21:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:21:46.588074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:46.589659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:46.640990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:46.695009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc3f0e-aea3-4a12-9954-550566afb797 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:46.768444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:46.769019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.769301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.769573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.771157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:46.870704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:46.872134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:46.875747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:46.889989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 752/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:21:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:21:46.935695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c7c7-329c-497f-b38d-f54449bd065e None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:46.938690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c7c7-329c-497f-b38d-f54449bd065e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:46.938690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00c7c7-329c-497f-b38d-f54449bd065e None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:46.938690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 754/1506] 10.222.0.27 () {52 vars in 830 bytes} [Mon Aug 8 02:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:21:46.941151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:46.942350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:47.043871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87b7d6-d713-4f71-8f66-62e924876e20 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:47.137212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:47.137841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:47.138120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:47.138391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] PATH_INFO: `/v3/users/65d8037802f241878a38c92d261e3616` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:47.150948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d8037802f241878a38c92d261e3616)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:47.151968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:47.151968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:21:47.151968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:21:47.153216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:47.163743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:47.393899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d8037802f241878a38c92d261e3616'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:47.410193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d8037802f241878a38c92d261e3616'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:47.436204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d8037802f241878a38c92d261e3616'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:47.436682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d8037802f241878a38c92d261e3616'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:47.437210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d8037802f241878a38c92d261e3616'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:47.497818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 753/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:21:46 2022] DELETE /identity/v3/users/65d8037802f241878a38c92d261e3616 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:21:47.503175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:47.508223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:47.508223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:47.560771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:21:47.561975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:47.636800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:47.651260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 755/1508] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:21:47.660022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:47.661295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:47.760793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cef50-063b-46ef-95af-7247cca84b32 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:47.816291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:47.835294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:47.835898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:47.836173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:47.836438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:47.838045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:47.945806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:47.947162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162a839-36a1-47e5-8693-eb8d0d59b42d service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:47.965362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 754/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:21:48.432916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:21:48.434521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:21:48.446617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9d49b-f0bd-4c6e-8946-e268713b08a2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:21:48.554714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:21:48.555348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:21:48.555625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:21:48.555893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] PATH_INFO: `/v3/projects/9fcc3b45fa7e4f4f90232766c498c5fb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:21:48.566648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcc3b45fa7e4f4f90232766c498c5fb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:21:48.567711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:21:48.567711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:21:48.567711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:21:48.568927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:21:48.711520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcc3b45fa7e4f4f90232766c498c5fb', 'request_id': 'req-5943a44a-8ee5-4bf8-adb3-a814d6b75009'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:48.768200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcc3b45fa7e4f4f90232766c498c5fb', 'request_id': 'req-5943a44a-8ee5-4bf8-adb3-a814d6b75009'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:48.775072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcc3b45fa7e4f4f90232766c498c5fb', 'request_id': 'req-5943a44a-8ee5-4bf8-adb3-a814d6b75009'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:48.809956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] The token cache is being invalidate because project 9fcc3b45fa7e4f4f90232766c498c5fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:21:48.810247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:48.810679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5943a44a-8ee5-4bf8-adb3-a814d6b75009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:21:48.812148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 756/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:21:48 2022] DELETE /identity/v3/projects/9fcc3b45fa7e4f4f90232766c498c5fb => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:21:57.963066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=62 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:21:58.807572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:17.575465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:17.576088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:17.576088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:17.577434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:17.582428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606a1c-b8c9-4002-8165-bc9d477f4d48 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:17.582860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606a1c-b8c9-4002-8165-bc9d477f4d48 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:17.582967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606a1c-b8c9-4002-8165-bc9d477f4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:17.664769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-03606a1c-b8c9-4002-8165-bc9d477f4d48 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:17.665971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-03606a1c-b8c9-4002-8165-bc9d477f4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:17.771333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:17.928638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:17.930809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:18.047070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238293a4-8f97-4b1b-b7af-fde165b25cf5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:18.052409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:18.054568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:18.054568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:18.054568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:18.063062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:18.064750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317deb54-f3fb-4a1b-9214-02f1d48cc914 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:18.075257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 756/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:23:18.913204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fc73-3f06-49c0-bd4f-81570a01c656 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:18.914100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fc73-3f06-49c0-bd4f-81570a01c656 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:18.914381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224fc73-3f06-49c0-bd4f-81570a01c656 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:18.915739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:18.921280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9641c-9ddd-402d-9f2b-c52def205701 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:18.921706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9641c-9ddd-402d-9f2b-c52def205701 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:18.921829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9641c-9ddd-402d-9f2b-c52def205701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:18.942869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-67d9641c-9ddd-402d-9f2b-c52def205701 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:18.944079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-67d9641c-9ddd-402d-9f2b-c52def205701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:18.952195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:19.104647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224fc73-3f06-49c0-bd4f-81570a01c656 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:19.120007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224fc73-3f06-49c0-bd4f-81570a01c656 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:19.125513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:19.126112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:19.126393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:19.126667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:19.135165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:19.136775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48412f42-8615-42f9-9c14-807c44d1c64e service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:19.146275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 759/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:19.718829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7d7f-d87e-4e20-9d70-3068764f0aac None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:19.719112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7d7f-d87e-4e20-9d70-3068764f0aac None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:19.719393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7d7f-d87e-4e20-9d70-3068764f0aac None None] PATH_INFO: `/v3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:19.720718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 758/1517] 10.222.0.45 () {54 vars in 925 bytes} [Mon Aug 8 02:23:19 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:23:19.726941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19ee-ad89-4907-a10e-a8ae30fbca52 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:19.727229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19ee-ad89-4907-a10e-a8ae30fbca52 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:19.727535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19ee-ad89-4907-a10e-a8ae30fbca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:19.787531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e44f19ee-ad89-4907-a10e-a8ae30fbca52 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:19.788784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e44f19ee-ad89-4907-a10e-a8ae30fbca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:19.805780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:19.860567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 760/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 8 02:23:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:23.770273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff32418-b156-47b0-ab9c-2b05dae4c91c None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:23.770876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff32418-b156-47b0-ab9c-2b05dae4c91c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:23.770876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff32418-b156-47b0-ab9c-2b05dae4c91c None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:23.772083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 759/1519] 10.222.0.27 () {54 vars in 961 bytes} [Mon Aug 8 02:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:23.777178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6057bc-f60a-4beb-a173-54e8ed074f89 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:23.777630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6057bc-f60a-4beb-a173-54e8ed074f89 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:23.777743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6057bc-f60a-4beb-a173-54e8ed074f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:23.797564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8a6057bc-f60a-4beb-a173-54e8ed074f89 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:23.798764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8a6057bc-f60a-4beb-a173-54e8ed074f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:23.807405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 761/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Aug 8 02:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:23.815389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff32418-b156-47b0-ab9c-2b05dae4c91c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:23.828948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff32418-b156-47b0-ab9c-2b05dae4c91c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:23.833758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:23.834334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:23.834607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:23.834874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:23.843386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:23.844967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a2d41-4971-47ed-8f0e-ebb18bb059d0 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:23.854218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 760/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:30.389763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0bdaa-c030-4ed9-afe3-ca9515e0b904 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.390375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0bdaa-c030-4ed9-afe3-ca9515e0b904 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.390375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0bdaa-c030-4ed9-afe3-ca9515e0b904 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.391671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:30.396469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131c0e-1ce5-495b-906d-499ff75acb27 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.397027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131c0e-1ce5-495b-906d-499ff75acb27 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.397027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c131c0e-1ce5-495b-906d-499ff75acb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.416683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1c131c0e-1ce5-495b-906d-499ff75acb27 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:30.417874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1c131c0e-1ce5-495b-906d-499ff75acb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:30.426170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:30.584241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0bdaa-c030-4ed9-afe3-ca9515e0b904 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:30.599615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0bdaa-c030-4ed9-afe3-ca9515e0b904 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:30.609564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:30.609564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:30.609564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:30.609564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:30.614540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:30.616113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396004e5-f8e1-4eb7-8762-2e80c21561ed service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:30.625597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 763/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:31.446548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ae80-22dd-4ccc-a56e-42dd3cc0409f None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:31.447091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ae80-22dd-4ccc-a56e-42dd3cc0409f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:31.447091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3ae80-22dd-4ccc-a56e-42dd3cc0409f None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:31.448357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 02:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:31.453698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc395d2c-07de-4969-8c97-ea3110582afa None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:31.454146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc395d2c-07de-4969-8c97-ea3110582afa None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:31.454266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc395d2c-07de-4969-8c97-ea3110582afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:31.474046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-dc395d2c-07de-4969-8c97-ea3110582afa None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:31.475238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dc395d2c-07de-4969-8c97-ea3110582afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:31.483121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 02:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:31.635362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3ae80-22dd-4ccc-a56e-42dd3cc0409f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:31.649917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3ae80-22dd-4ccc-a56e-42dd3cc0409f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:31.655873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:31.656452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:31.656753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:31.657023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:31.664831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:31.666392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5e64f-c56a-4d6e-9907-9d5a6f0fc8ff service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:31.675587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 763/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:35.703737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:35.704290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:35.704290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:35.705671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:35.710763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424098ae-aae1-45df-a74b-82021c659033 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:35.711163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424098ae-aae1-45df-a74b-82021c659033 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:35.711309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424098ae-aae1-45df-a74b-82021c659033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:35.731157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-424098ae-aae1-45df-a74b-82021c659033 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:35.733093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-424098ae-aae1-45df-a74b-82021c659033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:35.741610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:35.816301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:35.818033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:35.899976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72893e-56fd-4cb8-85a4-cd59cf407dbc None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:35.903427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:35.904641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:35.904944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:35.905254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:35.914156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:35.915574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac876752-982f-4120-beab-dee455d565f9 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:35.924726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 766/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:37.932454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66b6d4-140b-4caf-a384-ff4740189907 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:37.933039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66b6d4-140b-4caf-a384-ff4740189907 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:37.933039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66b6d4-140b-4caf-a384-ff4740189907 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:37.934599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:37.940174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92e72-5779-4484-bad4-76db85cc3904 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:37.940590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92e72-5779-4484-bad4-76db85cc3904 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:37.940783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92e72-5779-4484-bad4-76db85cc3904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:37.960508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-35c92e72-5779-4484-bad4-76db85cc3904 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:37.961759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-35c92e72-5779-4484-bad4-76db85cc3904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:37.969908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:38.005947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66b6d4-140b-4caf-a384-ff4740189907 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:38.011356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=75 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:38.018050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66b6d4-140b-4caf-a384-ff4740189907 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:38.021654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:38.025371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:38.025371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:38.025371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:38.031876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:38.033412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40c8e0-fd48-409d-9b31-837f3758dcba service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:38.042696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 766/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:41.496494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:41.497097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:41.497097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:41.498920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:41.504029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a573f3b-5ade-4e45-b02d-84274209913e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:41.504450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a573f3b-5ade-4e45-b02d-84274209913e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:41.504565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a573f3b-5ade-4e45-b02d-84274209913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:41.524225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7a573f3b-5ade-4e45-b02d-84274209913e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:41.525437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7a573f3b-5ade-4e45-b02d-84274209913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:41.533262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:41.840870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:41.843462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:41.908057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:41.920850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e5a7-cf48-44a0-882d-6b257dccaa80 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:41.928063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:41.928063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:41.928063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:41.928063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:41.934855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:41.936295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27067fa0-943b-4966-8fb4-a57f70359f1d service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:41.945514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 769/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:43.976108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:43.976622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:43.976622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:43.977982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 02:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:23:43.983177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30bab7-f856-495c-ab07-ec81eaabee2e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:43.983608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30bab7-f856-495c-ab07-ec81eaabee2e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:43.983713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30bab7-f856-495c-ab07-ec81eaabee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:44.003188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7b30bab7-f856-495c-ab07-ec81eaabee2e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:23:44.004386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7b30bab7-f856-495c-ab07-ec81eaabee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:44.012121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:23:44.313771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:23:44.315555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:23:44.328579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16987e5e-f1a6-4121-a705-5aa10b0ed513 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:23:44.332100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:23:44.332723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:23:44.333009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:23:44.333283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:23:44.342905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:23:44.344368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d00497-0fad-4571-990d-7521384156fd service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:23:44.353681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 769/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:23:53.998809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=11 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:23:54.349498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=14 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:08.425359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.427121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.427121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.453639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:08.455030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:08.466812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 771/1540] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:08.475514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d800-5c80-4e81-b9d4-9c0a7b82d5b5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.476062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d800-5c80-4e81-b9d4-9c0a7b82d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.476062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d800-5c80-4e81-b9d4-9c0a7b82d5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.507875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2f57d800-5c80-4e81-b9d4-9c0a7b82d5b5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:08.509106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2f57d800-5c80-4e81-b9d4-9c0a7b82d5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:08.520626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 770/1541] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:08.525463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.540414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d0772-1593-4a49-ae2e-cd7f7c352a52 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:08.549897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:08.550872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.551329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.551776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.552603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:08.553250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:08.555127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:08.569786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49388-c543-4190-a50f-862246f70f13 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.570046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49388-c543-4190-a50f-862246f70f13 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.570325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a49388-c543-4190-a50f-862246f70f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.574373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 772/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:08.595641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.608082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-86a49388-c543-4190-a50f-862246f70f13 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:08.612693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-86a49388-c543-4190-a50f-862246f70f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:08.613772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a90b6b-3895-4442-adee-4bdd6d4b416a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:08.625463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:08.626394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.626682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 771/1543] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:08.627040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.627485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.628321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:08.629726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:08.629726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:25:08.629726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:25:08.631347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:08.635525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.635820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.636098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.674079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:08.675516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:08.687419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 772/1544] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:08.709915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.719838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a4d4d-7987-47a1-8f55-dadaa3ba398e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:08.728854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:08.729469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.729749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.730091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.730747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:08.731055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:08.732675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:08.732675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:25:08.732675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:25:08.733692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:08.751177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 773/1545] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:25:08.760724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.772057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456247-4e29-497d-a8d1-6bf930b4439c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:08.780851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:08.781474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:08.781757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:08.782032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:08.782680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:08.784700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:08.802454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 773/1546] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:08.807203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.893495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 774/1547] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:08.900052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:08.901558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e1a5e-4544-4e57-ad70-59f62f93ba1b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:08.996143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802d562-cb19-492a-84c5-c085ac6310b2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.006593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.007231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.007511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.007783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.008428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.009398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.010018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.010330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.010611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.011408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.012441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:09.012441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:25:09.012441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:25:09.014244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d79baf-9bd5-48bd-88af-c6cb7e328789 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.018583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2091d-d058-45d4-87ac-aee1e7a4627a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.137653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 774/1548] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/users => generated 292 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:09.139704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.140031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.140309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.162397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 775/1549] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:08 2022] POST /identity/v3/users => generated 296 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:09.166346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225364-8e60-4197-aa1d-1eb27ed10b64 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.166346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225364-8e60-4197-aa1d-1eb27ed10b64 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.166346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225364-8e60-4197-aa1d-1eb27ed10b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.179208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.180615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.190831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 775/1550] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.191991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.205042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb6b9b-c524-43de-9640-ad2d97c39bdf None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.207539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e7225364-8e60-4197-aa1d-1eb27ed10b64 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.209320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e7225364-8e60-4197-aa1d-1eb27ed10b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.210336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.211058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.211343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.211871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.212519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.214346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.221569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 776/1551] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.223537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbacc9-161e-47bc-a164-7f445a3fb3e1 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.224064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbacc9-161e-47bc-a164-7f445a3fb3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.224550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbacc9-161e-47bc-a164-7f445a3fb3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.235611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 776/1552] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:09.236336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.248427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95683553-b083-4cf1-b0e6-6d8bc0755bfd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.251032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3fcbacc9-161e-47bc-a164-7f445a3fb3e1 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.252963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3fcbacc9-161e-47bc-a164-7f445a3fb3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.253617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.254335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.254611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.254888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.255528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.257164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce5ecc-4baa-4ad2-a17f-06cdb8e45b3a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.264830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 777/1553] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.267175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1311e-7a3e-4e12-9183-9da886954a9f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.267688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1311e-7a3e-4e12-9183-9da886954a9f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.268183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1311e-7a3e-4e12-9183-9da886954a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.273079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 777/1554] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:09.277394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.277687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.277965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.304824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f0a1311e-7a3e-4e12-9183-9da886954a9f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.306471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f0a1311e-7a3e-4e12-9183-9da886954a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.310785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.311964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.315084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 778/1555] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.316421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.316733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.316997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.320365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 778/1556] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.321056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.330615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690390a-d89e-4e3e-a098-1dfc9cc4b221 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.335388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.335986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.336268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.336541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.337212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.337962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.338812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101938c-d8a1-4c94-bbcc-09a2114dcc3b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.339139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.345820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:09.347975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 779/1557] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.348618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.353802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 779/1558] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:09.353802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30eac-9e10-4f31-a4da-ace0bb328f25 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.353802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30eac-9e10-4f31-a4da-ace0bb328f25 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.353802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30eac-9e10-4f31-a4da-ace0bb328f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.359433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0578-8807-4134-8241-e3bc2cce3d2b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.363938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.364541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.364850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.365124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.365774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.367357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c39edf-a7e0-4bb3-bfef-80bea52424ac admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.372574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3bf30eac-9e10-4f31-a4da-ace0bb328f25 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.373851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3bf30eac-9e10-4f31-a4da-ace0bb328f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.377887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 780/1559] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:09.379126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.379411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.379685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.385415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 780/1560] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.387349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198d0d1-7257-499b-abc0-1fb25ce327b0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.387754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198d0d1-7257-499b-abc0-1fb25ce327b0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.388862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198d0d1-7257-499b-abc0-1fb25ce327b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.399032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.400232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.408139 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9198d0d1-7257-499b-abc0-1fb25ce327b0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.408785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 781/1561] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.409368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9198d0d1-7257-499b-abc0-1fb25ce327b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.416497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.417497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 781/1562] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.417497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.417497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.417497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.429984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfd28c-a5b9-47ed-87f1-c3579e835af4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.435481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.436085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.436360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.436634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.437316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:09.437616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.439196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.440482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.441713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.452085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 782/1563] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:09.453053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 782/1564] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.453669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.454077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.463625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48e956-4470-4f2d-9e3c-1b32cc307ecc admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.463794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d6478-d1c1-47bc-a321-4008f3b2068c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.468356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.468519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.469050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.469153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.469323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.469432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.469594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.469697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] PATH_INFO: `/v3/projects/ca29b2b9a59d4826bfd25f4dcfa9cffe/users/578cd837d9fe442587f8c6c8528334c0/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.470233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:09.470533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.472106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.484645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 783/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:09.485563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.486551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca29b2b9a59d4826bfd25f4dcfa9cffe, user_id=578cd837d9fe442587f8c6c8528334c0, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.487597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:09.487597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:25:09.487597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:25:09.489601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.496949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6491d3-3fa8-45a6-969c-aeacdaf27b5f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.502584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.503167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.503443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.503714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] PATH_INFO: `/v3/projects/8af7d2bbc6d94273bd39c81423d8cebe/users/5c8ae69712c14878bd2aab6d33512b7a/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.518133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af7d2bbc6d94273bd39c81423d8cebe, user_id=5c8ae69712c14878bd2aab6d33512b7a, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.520832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64a0d-40b5-49cc-afb7-f5f177b8e6b1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.559009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 783/1566] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:09 2022] PUT /identity/v3/projects/ca29b2b9a59d4826bfd25f4dcfa9cffe/users/578cd837d9fe442587f8c6c8528334c0/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:09.561084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.583165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:09.588068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 784/1567] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:09 2022] PUT /identity/v3/projects/8af7d2bbc6d94273bd39c81423d8cebe/users/5c8ae69712c14878bd2aab6d33512b7a/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:09.590807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8373e-d139-4e03-a0a8-165d5057c221 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.591000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8373e-d139-4e03-a0a8-165d5057c221 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.591382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8373e-d139-4e03-a0a8-165d5057c221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.612620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5ea8373e-d139-4e03-a0a8-165d5057c221 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.613860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5ea8373e-d139-4e03-a0a8-165d5057c221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.631311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bb26c-8d8e-486e-bf0d-22457694f36e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.678553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 785/1568] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.679913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.680539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.681018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.690881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.693732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.693732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.693732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.693732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:09.693732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.694677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.701463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.702767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.706404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 784/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:09.707209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.711614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 786/1570] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.712225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.721874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934052-98ce-4602-bb31-c9d8815e3179 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.722508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be02b20-4f33-4b58-aefe-cad93da4f44e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.726520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.728484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.728633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.728800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.728800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.728997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.728997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.729165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.729252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.729252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:09.729449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.730260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea8bff-06d1-42e6-9312-52d32a0f8a95 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.730899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.744215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 785/1571] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:09.745031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.756489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a693379-e4d9-4dde-9f35-e6160a289ce9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.761021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.761605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.761884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.762897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.762897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.764750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d6204-5f6b-41cf-b1de-7fa836f93c24 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:09.828902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 787/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:09.830196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.830482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.830758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.849942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.851181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.895425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 786/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:09.896922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.897276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.897628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.914086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 788/1574] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.914909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.916772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:09.918316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:09.924856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af059e3-bcaf-4648-a6ba-f5e0f6e4d8f0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:09.937495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:09.984853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:09.985494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:09.985839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 787/1575] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:09.985960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:09.986128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:09.986733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:09.986842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:09.987103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:09.989015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.006400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 789/1576] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:10.006400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.006726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e368adf-e6dd-4629-bd1b-a354bd7250c0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.010231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.010845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.011122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.011392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.012044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:10.012346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.014648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.015706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f67e9-759e-457a-9be1-9c63ec21f8e2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.020307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.020938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.021228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.021544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.022173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.024087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.026682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 788/1577] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:10.028127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.030701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:10.038014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca179d4-eb74-4078-b7f4-db2de4500841 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.042548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.043171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.043446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.043718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.044407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.046851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.194423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 790/1578] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.195382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.209664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 789/1579] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.210462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.268095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513886cc-30e1-45cd-b809-6077150e77b5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.272696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.273299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.273585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.273893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.274222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab513-7c1f-464e-87ad-e2a9525e1e32 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.274514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.276524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.278872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.279478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.279755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.280025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.280645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.282661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.461741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 791/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 295 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.462926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.464590 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:10.502700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 790/1581] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.504110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.505868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:10.521420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b494892-4a3a-4a7c-be23-4138d4387334 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.524982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.525588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.525869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.526143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.535723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.538442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.547253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 792/1582] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:10.548034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.557482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980c80-e2c8-4757-be2a-c331a62bd959 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.562682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.563289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.563562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.564388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.565045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.567354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fb39-2603-4532-9f5b-c5f773bdcb95 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.567587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.571756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.572296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.572579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.572886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.582477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.584978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.594862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 791/1583] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:10.595784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.605288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1f584-b278-44c8-b65b-ed290c7d289f service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.610524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.619513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.619513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.619513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.619513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.619513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.675359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 793/1584] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.676081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.734729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:10.736318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:10.740487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec458a1-6583-4d40-9982-b9a9aa99d778 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.745338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 792/1585] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.748776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.748776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.748776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.748776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.749129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.749834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.750395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.814611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6d442-3915-4132-923d-10566f3c418b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.819377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.819990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.820273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.820546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.821397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.823331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.854005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 794/1586] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.854005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.861775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954205e1-2088-498d-99af-9b31a77f170b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.866634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.867234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.867511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.867783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.868427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.870049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.880967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 795/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:10.892517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.892517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b45507-a9d1-429f-9300-d61bd57bbf81 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.896356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.896965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.897265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.897536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.898170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.899750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.911209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 796/1588] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:10.911933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.921602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6e219-3820-4e1c-ac3f-67bd4e952fc7 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.927114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.927905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.928321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.928650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.929378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.931331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:10.956974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 793/1589] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:10.956974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:10.966115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d41ff-11cf-4747-a82d-fde0e599adbf admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:10.971582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:10.972178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:10.972555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:10.972874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:10.973610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:10.975527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.067256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 797/1590] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 303 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:11.067256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.073819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6ea96-e085-463d-91ac-140810b8977d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.079230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.079830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.080104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.080375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.081097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.082719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.093351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 798/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.094231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.106182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 794/1592] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:10 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:11.106182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.111578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088701a0-a9b3-46c1-80e0-82b215cc5f09 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.116005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.116707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.116939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47247b38-39da-4035-83d9-09c14903e752 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.117120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.117461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.118170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.119882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.122055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.123547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.123968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.124241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.125038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.126682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.133850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 799/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.134688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.142724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 795/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.145755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a8a9-c550-4196-a588-12ab8bfcbd03 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.146708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.150598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.151328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.151683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.152088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.152873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.159083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.159256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb00da-e3a9-4ff5-ac17-11f39c40d87a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.162392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.162794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.163207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.163505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.164212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.165885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.167788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 800/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.168552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.179689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2b83-b647-4619-821a-56b39dd22c2d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.185427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 796/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.187017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.189538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.193994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.193994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.193994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.193994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.193994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.197390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993d814-f2f1-4058-a9d5-c7bc9f775caa admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.203039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.203652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.204384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.204384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.204877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.206454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.211918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 801/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.211918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.220558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832471e-7ec2-4ea2-aedd-5f00de557a96 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.225568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 797/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.225568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.227583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.228363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.228716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.234141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] PATH_INFO: `/v3/projects/7caa43bf43ad4e84b4b648e9ab33bd7f/users/d451fb2f26ce4f50b010fb2a387c4bf3/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.235689 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c28b9c-086a-4f7c-922f-72ecbf465c3a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.240712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.241406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.241694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.241966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] PATH_INFO: `/v3/projects/9c3c108c17cf43f1b0ed3459b8434717/users/13b4614279db45fb8f582ac0a21c829d/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.246599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7caa43bf43ad4e84b4b648e9ab33bd7f, user_id=d451fb2f26ce4f50b010fb2a387c4bf3, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.249751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.258160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3c108c17cf43f1b0ed3459b8434717, user_id=13b4614279db45fb8f582ac0a21c829d, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.260854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.381819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 802/1599] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/7caa43bf43ad4e84b4b648e9ab33bd7f/users/d451fb2f26ce4f50b010fb2a387c4bf3/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:11.389535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.430033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 798/1600] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/9c3c108c17cf43f1b0ed3459b8434717/users/13b4614279db45fb8f582ac0a21c829d/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:11.430033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.463959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc6326c-a622-4095-ad6b-af484fa1573f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.509533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:11.524050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c762af-8a62-4f09-a91d-0c838f5a8f53 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.529652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.530304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.530588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.530867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.531575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.533333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.540986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.542779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.543555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.544041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.549284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 799/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.549284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.551826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.553593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.561143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759efd56-62b6-433f-9c21-a86d85f02f28 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.566136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 803/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.567018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.569866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.570562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.570925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.571258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] PATH_INFO: `/v3/projects/ee8baa7d6e294e27ae2fb12eded3709d/users/eb4c92edcb024e32a549650eb85f6952/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.574816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:11.580870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14bb69-7d86-4a9c-884b-7221db858f5a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.590071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.590888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.591823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.592051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8baa7d6e294e27ae2fb12eded3709d, user_id=eb4c92edcb024e32a549650eb85f6952, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.592924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.593814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.595227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.595860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735aa8bf-4553-4293-9725-c06ad7cc6410 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.606562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 804/1603] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:11.609950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67b66-6e3a-495b-aa29-de82b2fa33fc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.609950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67b66-6e3a-495b-aa29-de82b2fa33fc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.609950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef67b66-6e3a-495b-aa29-de82b2fa33fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.627954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-8ef67b66-6e3a-495b-aa29-de82b2fa33fc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:11.629112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8ef67b66-6e3a-495b-aa29-de82b2fa33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:11.637706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 805/1604] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:11.639699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.639979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.641357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.666749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] MFA Rules not processed for user `13b4614279db45fb8f582ac0a21c829d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:11.667951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:11.705840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 800/1605] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/ee8baa7d6e294e27ae2fb12eded3709d/users/eb4c92edcb024e32a549650eb85f6952/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:11.707594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.746158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 806/1606] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:11.747175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.766720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9aed3-1f56-4bc7-ad7d-8907a2489daf None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.808889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99382cf-a022-4c29-9aa3-2fec16a0f02c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:11.817357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.818337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.818673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.818948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] PATH_INFO: `/v3/projects/d7025c9928ff43feb51778ce5724e0ec/users/8a43738760144d429a545e9a18188b7c/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.836100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7025c9928ff43feb51778ce5724e0ec, user_id=8a43738760144d429a545e9a18188b7c, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.838978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.851932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:11.852888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:11.853228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:11.853563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] PATH_INFO: `/v3/projects/5515114a058944a49182063213de54d3/users/2aeb52d51d3f44319bb4e114858222f0/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:11.869059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5515114a058944a49182063213de54d3, user_id=2aeb52d51d3f44319bb4e114858222f0, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:11.871466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:11.913302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 801/1607] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/d7025c9928ff43feb51778ce5724e0ec/users/8a43738760144d429a545e9a18188b7c/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:11.914114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.961424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 807/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/5515114a058944a49182063213de54d3/users/2aeb52d51d3f44319bb4e114858222f0/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:11.961424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:11.984061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1b5af-6557-4b80-85b1-2bfd6fcc5d94 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.004103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:12.032165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9ea9-ff38-4299-b4b7-0a1db369d843 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.044382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.045462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.045680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.045931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.057881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.062644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.062644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.062644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] PATH_INFO: `/v3/projects/4b26f1294be945b1bfb2e5e0b47cc2b3/users/e223926a7d054350a86ba5943908737a/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.079572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b26f1294be945b1bfb2e5e0b47cc2b3, user_id=e223926a7d054350a86ba5943908737a, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.082451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae13ab-bcd5-4b31-bc5b-103be199175a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.111284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.112852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2051a56-dd60-4f2f-8ef0-94dac58d8900 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.123603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 808/1609] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:12.125046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.125397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.125680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.157623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:12.159725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:12.168652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 809/1610] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:12.169911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.179383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:12.181830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fad7ea-b9fc-499d-a481-b9212f7363e4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.186374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 802/1611] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:11 2022] PUT /identity/v3/projects/4b26f1294be945b1bfb2e5e0b47cc2b3/users/e223926a7d054350a86ba5943908737a/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:12.187627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecba859-00c7-43bc-a19d-0f881a3cfc03 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.187913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecba859-00c7-43bc-a19d-0f881a3cfc03 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.188243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecba859-00c7-43bc-a19d-0f881a3cfc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.215474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0ecba859-00c7-43bc-a19d-0f881a3cfc03 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:12.216990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0ecba859-00c7-43bc-a19d-0f881a3cfc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:12.261698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.262323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.262933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.263806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.264695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.266833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58feba3a-bb3a-417f-93cc-a3c44655cc89 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.301039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 803/1612] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:12.303006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.303458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.305223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.333114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 804/1613] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:12.356473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.418187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 810/1614] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/projects => generated 363 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:12.423241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.423241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.423241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.423519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e115cad-2a01-46db-98c0-ee040d65ecc7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.452782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] MFA Rules not processed for user `e223926a7d054350a86ba5943908737a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:12.453995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:12.503830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.504625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.514595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.514595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.584621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 811/1615] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:12.584621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.601906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7cea-1cce-4611-9abf-618a9075e714 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.620539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.621403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.622403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.622521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.622817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.623171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.639339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.641837 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.650678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 805/1616] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:12.654435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.655609 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 812/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:12.656736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.664824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab575-eeff-4882-a969-9cae694c0922 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.669688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec5d3b-29f3-4bd1-9ec3-2f2bc00e0165 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.669997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.670723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.671005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.671276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.687044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.687044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.687044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.687044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.687532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.687532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.687702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.687702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.699659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 806/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:12.701055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.764857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad8da4-2d15-4ab9-8e8f-adb351c55cee service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.768697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.769324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.769602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.776041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.778695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.780155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.789815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 807/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:12.843710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:12.852647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 813/1620] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:12 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:12.858231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.874072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62478d-ca31-4249-8470-0bdbf7255e73 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.886899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.888154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.888437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.888732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:12.889391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:12.894782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:12.939644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 808/1621] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:12.939779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:12.953358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440ebb9-9218-4824-a6eb-7361a5e386c5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:12.970088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:12.971175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:12.971509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:12.975798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] PATH_INFO: `/v3/projects/52630a77455c495a8286944ce83a46e0/users/41cb31702dbc411c8e3ba9cb32fb9b66/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:13.011152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52630a77455c495a8286944ce83a46e0, user_id=41cb31702dbc411c8e3ba9cb32fb9b66, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:13.016247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:13.059406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:13.139719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 814/1622] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:12 2022] PUT /identity/v3/projects/52630a77455c495a8286944ce83a46e0/users/41cb31702dbc411c8e3ba9cb32fb9b66/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:13.141008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:13.153242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfef3c-0409-43aa-96e3-464485a57e9e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:13.285197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:13.285840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:13.286127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:13.287396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:13.289174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:13.292388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556c881-0211-4c14-a7b8-2b0ebcc2b642 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:13.322287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 809/1623] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:13.323831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:13.338436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf68cb-9b93-4bd0-bf8c-870a947e19d5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:13.347429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:13.348302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:13.348701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:13.350026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] PATH_INFO: `/v3/projects/52630a77455c495a8286944ce83a46e0/users/41cb31702dbc411c8e3ba9cb32fb9b66/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:13.357532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] RBAC: Authorizing `identity:create_grant(project_id=52630a77455c495a8286944ce83a46e0, user_id=41cb31702dbc411c8e3ba9cb32fb9b66, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:13.361536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:13.387764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 815/1624] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:13 2022] PUT /identity/v3/projects/52630a77455c495a8286944ce83a46e0/users/41cb31702dbc411c8e3ba9cb32fb9b66/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:13.391471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:13.392029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:13.392517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:13.428581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] MFA Rules not processed for user `41cb31702dbc411c8e3ba9cb32fb9b66`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:13.431617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:13.569370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 810/1625] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:13.594130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:13.613275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:13.697917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23183b9-7924-4aae-ab12-3052e150017d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:13.697917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:13.697917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:13.697917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:13.698402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:13.698402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:13.698402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3854dc-db46-425d-bf4a-941c6c85957f service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:13.700077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 816/1626] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:15.534874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:15.628384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfaec2-da4e-4e75-a12b-9bcfab50ffc0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:15.634692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:15.636705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:15.636705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:15.638352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] PATH_INFO: `/v3/users/13b4614279db45fb8f582ac0a21c829d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:15.642990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b4614279db45fb8f582ac0a21c829d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:15.646002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: WARNING py.warnings [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:15.646002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: warnings.warn(msg) Aug 08 02:25:15.646002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: Aug 08 02:25:15.662190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:15.965118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b4614279db45fb8f582ac0a21c829d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:16.009119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b4614279db45fb8f582ac0a21c829d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:16.043142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b4614279db45fb8f582ac0a21c829d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:16.044895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b4614279db45fb8f582ac0a21c829d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:16.046185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b4614279db45fb8f582ac0a21c829d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:16.183590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 811/1627] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:15 2022] DELETE /identity/v3/users/13b4614279db45fb8f582ac0a21c829d => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:16.189530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3ad5-754e-41a7-97f0-a42219c2d59d None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:16.190076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3ad5-754e-41a7-97f0-a42219c2d59d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:16.190076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c3ad5-754e-41a7-97f0-a42219c2d59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:16.243522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bc6c3ad5-754e-41a7-97f0-a42219c2d59d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:16.246246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bc6c3ad5-754e-41a7-97f0-a42219c2d59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:16.432478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 817/1628] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:16.746660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:16.749362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:16.893433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adca913-9867-4e3b-a3aa-d8f96d161b0c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:16.899345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:16.900290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:16.900774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:16.901200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:16.913191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:16.914972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2a3a0-0769-4c82-a45f-62b1944eec0b service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:16.927610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 812/1629] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:16.947730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:16.947730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:16.947730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.001190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] MFA Rules not processed for user `8a43738760144d429a545e9a18188b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:17.004921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:17.035319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f9e1-a1a0-45a4-a3a3-2f89ebf3d2a6 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.035771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f9e1-a1a0-45a4-a3a3-2f89ebf3d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.036692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f9e1-a1a0-45a4-a3a3-2f89ebf3d2a6 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.039276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 813/1630] 10.222.0.27 () {54 vars in 962 bytes} [Mon Aug 8 02:25:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:17.053898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.054695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfd7ae-9746-4b05-a143-69137c87e49e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.055143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfd7ae-9746-4b05-a143-69137c87e49e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.055745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfd7ae-9746-4b05-a143-69137c87e49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.148179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:17.160043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-97cfd7ae-9746-4b05-a143-69137c87e49e None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:17.162840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-97cfd7ae-9746-4b05-a143-69137c87e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:17.166734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 818/1631] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:16 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:17.172123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:17.173718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:17.195547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e38da1-93ee-4d3d-901a-019f16f21fca None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:17.201367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:17.201865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.202212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.202552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.203267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:17.205766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:17.264259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 814/1632] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:17.269805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366e2d-aab5-425d-8a25-0e927d6d1561 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.270395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366e2d-aab5-425d-8a25-0e927d6d1561 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.270834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366e2d-aab5-425d-8a25-0e927d6d1561 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.272614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 815/1633] 10.222.0.27 () {54 vars in 912 bytes} [Mon Aug 8 02:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:25:17.278740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.291346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.291346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.322915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:17.325693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:17.326621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 819/1634] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:17 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:17.330396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:17.428797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 816/1635] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:17.458017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:17.466665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f95-f2a2-421c-b2a6-f5e303b5fea5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:17.467659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:17.472733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:17.480639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.480639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.480639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.480639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:17.480639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:17.569702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 820/1636] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:17 2022] POST /identity/v3/users => generated 307 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:17.576001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:17.625475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1c53-fe44-48f3-bee8-64d1fadb2b97 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:17.645952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:17.649521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.650174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.650174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.655643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:17.660809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:17.661246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75dc39-0877-4c34-9343-a9d53b67ad75 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:17.677141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:17.682309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.684498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.687309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.707510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 821/1637] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:17.712479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:17.730210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:17.736983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:17.765213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f017-de98-4aa1-a137-77c73e0085ef admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:17.780074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:17.780628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:17.783897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:17.784197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] PATH_INFO: `/v3/projects/192d598b1b0e4ebcbb22ddbb40546624/users/07405bc5defb4eb5a2e18891a549bc50/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:17.805561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 817/1638] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:17.853345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192d598b1b0e4ebcbb22ddbb40546624, user_id=07405bc5defb4eb5a2e18891a549bc50, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:17.857858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:17.984251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 822/1639] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:17 2022] PUT /identity/v3/projects/192d598b1b0e4ebcbb22ddbb40546624/users/07405bc5defb4eb5a2e18891a549bc50/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:17.988416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:18.009475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5928d-a5dd-49f7-82db-140f12ff0b12 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:18.132188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:18.132950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:18.133307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:18.133920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:18.134747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:18.138242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:18.163929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 818/1640] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:18.165842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:18.186716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe708d-79fa-48de-aa84-d0e40966eb83 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:18.193279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:18.193965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:18.194344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:18.194515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] PATH_INFO: `/v3/projects/192d598b1b0e4ebcbb22ddbb40546624/users/07405bc5defb4eb5a2e18891a549bc50/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:18.197258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=192d598b1b0e4ebcbb22ddbb40546624, user_id=07405bc5defb4eb5a2e18891a549bc50, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:18.206619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:18.238833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 823/1641] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:18 2022] PUT /identity/v3/projects/192d598b1b0e4ebcbb22ddbb40546624/users/07405bc5defb4eb5a2e18891a549bc50/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:18.750673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:18.752295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:18.770657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcfc9-a8b7-4499-bfab-c2f0cd0edbb8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:18.881904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:18.882774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:18.883838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:18.884572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] PATH_INFO: `/v3/projects/9c3c108c17cf43f1b0ed3459b8434717` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:18.892513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3c108c17cf43f1b0ed3459b8434717)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:18.895125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:19.157680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3c108c17cf43f1b0ed3459b8434717', 'request_id': 'req-fed7078e-e73c-40b2-9a49-16aed897c25d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.333032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3c108c17cf43f1b0ed3459b8434717', 'request_id': 'req-fed7078e-e73c-40b2-9a49-16aed897c25d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.340296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3c108c17cf43f1b0ed3459b8434717', 'request_id': 'req-fed7078e-e73c-40b2-9a49-16aed897c25d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.375674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] The token cache is being invalidate because project 9c3c108c17cf43f1b0ed3459b8434717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:19.375674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.375674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fed7078e-e73c-40b2-9a49-16aed897c25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.392760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 819/1642] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:18 2022] DELETE /identity/v3/projects/9c3c108c17cf43f1b0ed3459b8434717 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:19.397275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:19.399453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:19.427694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d95a78-7859-4842-96a9-b6c33b598a1d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:19.573787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:19.574447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:19.574721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:19.574993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] PATH_INFO: `/v3/users/41cb31702dbc411c8e3ba9cb32fb9b66` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:19.584277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cb31702dbc411c8e3ba9cb32fb9b66)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:19.584277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:19.584879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:19.867265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cb31702dbc411c8e3ba9cb32fb9b66'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.884600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cb31702dbc411c8e3ba9cb32fb9b66'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.902520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cb31702dbc411c8e3ba9cb32fb9b66'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.905976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cb31702dbc411c8e3ba9cb32fb9b66'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:19.907019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e740c9ea-f414-4e76-9702-3e4516e4eeef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cb31702dbc411c8e3ba9cb32fb9b66'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:20.032188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 824/1643] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:19 2022] DELETE /identity/v3/users/41cb31702dbc411c8e3ba9cb32fb9b66 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:20.698813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a624-8e62-4f08-bcc5-4bdadaa97474 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.699582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a624-8e62-4f08-bcc5-4bdadaa97474 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.700008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2a624-8e62-4f08-bcc5-4bdadaa97474 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.701903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 820/1644] 10.222.0.27 () {54 vars in 962 bytes} [Mon Aug 8 02:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:20.751791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0be84c-df04-4b00-a9f2-abb07ef85e2b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.752327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0be84c-df04-4b00-a9f2-abb07ef85e2b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.752327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0be84c-df04-4b00-a9f2-abb07ef85e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.786699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0a0be84c-df04-4b00-a9f2-abb07ef85e2b None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:20.788074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0a0be84c-df04-4b00-a9f2-abb07ef85e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:20.935876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 825/1645] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Aug 8 02:25:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:20.941865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.942138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.942428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.944053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 821/1646] 10.222.0.27 () {54 vars in 912 bytes} [Mon Aug 8 02:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:20.950976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:20.951899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:20.952183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:20.984271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:20.989803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.008457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 826/1647] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:21.022486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:21.024608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:21.128520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea9712-c93d-4a35-996b-58a831fcfa43 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:21.145333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:21.146397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:21.146825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:21.147255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:21.167691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:21.170175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe363a4-d5c5-44f3-9b4f-b475208af9b5 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:21.192727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 822/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:21.322002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.371660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:22.373236 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:22.608126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cbcc3-4f23-43fb-89eb-459d339805e6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:22.613372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:22.614293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.614734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.615170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] PATH_INFO: `/v3/projects/52630a77455c495a8286944ce83a46e0` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.623087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52630a77455c495a8286944ce83a46e0)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:22.625181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: WARNING py.warnings [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 02:25:22.625181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: warnings.warn(msg) Aug 08 02:25:22.625181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: Aug 08 02:25:22.627550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:22.678781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0e96-7cb7-4ea3-86aa-348d259e16c0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:22.681182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0e96-7cb7-4ea3-86aa-348d259e16c0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:22.681676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad0e96-7cb7-4ea3-86aa-348d259e16c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:22.704072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:22.752592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-65ad0e96-7cb7-4ea3-86aa-348d259e16c0 None None] MFA Rules not processed for user `07405bc5defb4eb5a2e18891a549bc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:22.752592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-65ad0e96-7cb7-4ea3-86aa-348d259e16c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:22.914267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52630a77455c495a8286944ce83a46e0', 'request_id': 'req-eac702dd-4c1d-4d7e-ae9b-878688a363c0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:22.925991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 823/1649] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:22.943775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:23.016024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52630a77455c495a8286944ce83a46e0', 'request_id': 'req-eac702dd-4c1d-4d7e-ae9b-878688a363c0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:23.023137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52630a77455c495a8286944ce83a46e0', 'request_id': 'req-eac702dd-4c1d-4d7e-ae9b-878688a363c0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:23.047665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] The token cache is being invalidate because project 52630a77455c495a8286944ce83a46e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:23.047965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:23.051767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-eac702dd-4c1d-4d7e-ae9b-878688a363c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:23.062808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 827/1650] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:22 2022] DELETE /identity/v3/projects/52630a77455c495a8286944ce83a46e0 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:23.070569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.070569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.070569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.103978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.114247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.237504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 824/1651] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.245956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.246226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.246683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.290088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:23.291482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:23.324366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 828/1652] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:23.331158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.346523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82ef6d-aba8-46d1-aa5e-1834125bfda4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.352844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.354308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.354640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.354926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.355967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:23.356432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.363791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.384583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 825/1653] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:23.386210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.400027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ccaf-7493-41c0-a1eb-ba17ad08d3b6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.412704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.413339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.413621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.413898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.414617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.416502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.608567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 829/1654] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/projects => generated 355 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:23.611590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.635075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14febd-c705-4d2e-95ec-76357f9007d9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:23.799856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:23.800534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:23.800534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:23.801227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:23.802332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:23.808188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:23.961784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 826/1655] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:23 2022] POST /identity/v3/users => generated 278 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:23.966046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:23.982714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176ad4-17ae-436e-a9fa-547472933ef5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.001274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.001946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.002234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.002526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.003398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.005199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.033665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 830/1656] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:24.036259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.053201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac921-4c88-42ef-9fe2-b8fcd8f52aa5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.069207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.074327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.074857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.075337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.076186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.078096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb7be-9e93-468b-850d-27fd22100294 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.121348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.127477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 827/1657] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:24.145160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f61703-ba08-4aa5-a590-99caf681ec77 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.166470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.167945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.168464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.169579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] PATH_INFO: `/v3/projects/eb31af0de3e7407b88224e5d7fd78fb2/users/a88a4a28263d40f499c1d5f5a78265fb/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.219719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb31af0de3e7407b88224e5d7fd78fb2, user_id=a88a4a28263d40f499c1d5f5a78265fb, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.222759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.359713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 831/1658] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:24 2022] PUT /identity/v3/projects/eb31af0de3e7407b88224e5d7fd78fb2/users/a88a4a28263d40f499c1d5f5a78265fb/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:24.361462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30626e-0a1f-47da-bbbb-ec5918b78cd2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.361746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30626e-0a1f-47da-bbbb-ec5918b78cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.362015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30626e-0a1f-47da-bbbb-ec5918b78cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.414574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2d30626e-0a1f-47da-bbbb-ec5918b78cd2 None None] MFA Rules not processed for user `a88a4a28263d40f499c1d5f5a78265fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:24.415801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2d30626e-0a1f-47da-bbbb-ec5918b78cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:24.576394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 828/1659] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:24.581104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:24.727999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae46e1c-3f5b-49d9-8607-833dbcfc9436 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:24.744789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:24.745671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.746180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.746906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.748714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:24.751577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:24.781669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:24.843618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:24.844063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:24.846894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:24.927351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 832/1660] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/projects => generated 355 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:24.935171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] MFA Rules not processed for user `578cd837d9fe442587f8c6c8528334c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:24.935171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:24.936906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.061514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90f7da-a87c-42fe-928c-e44211bfe71e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.069794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:25.077401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.078571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.078571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.078741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.079435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.085086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.156658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 829/1661] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:25.173574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.175788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:25.215296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 833/1662] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:24 2022] POST /identity/v3/users => generated 284 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:25.217643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.245632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f730b-4b6c-4e00-a291-3544d30a4663 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.261542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.279371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.279371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.279371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.279371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.279371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.322281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 834/1663] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:25.322281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.340891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd020b6-e08d-4132-a18a-539081d603ef admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.355507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.356163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.376899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.377677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] PATH_INFO: `/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4/users/3051d2e633eb4f2599365c6125356663/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.405815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3864a18f1e624ce3b61cd456df5fc2b4, user_id=3051d2e633eb4f2599365c6125356663, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.410786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.424519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b5f32-0207-4466-b98f-bb075b6228c6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.449398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.458253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.459924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.460213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.500992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.504210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.525426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 835/1664] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:25 2022] PUT /identity/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4/users/3051d2e633eb4f2599365c6125356663/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:25.531340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.542610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 830/1665] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:25.692028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243320f-0592-4df8-bbb0-9623d5b4100a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.701930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.702577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.702858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.703131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.704227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.706044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2628199-c6cc-497e-9435-a34512215858 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.727238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 836/1666] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:25.732321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:25.758254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11399c4-7d0a-4c80-9e6e-f0f7695fe8b4 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:25.772590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:25.774671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.775228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.775507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] PATH_INFO: `/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4/users/3051d2e633eb4f2599365c6125356663/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.783563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3864a18f1e624ce3b61cd456df5fc2b4, user_id=3051d2e633eb4f2599365c6125356663, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:25.792153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:25.835455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 831/1667] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:25 2022] PUT /identity/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4/users/3051d2e633eb4f2599365c6125356663/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:25.836262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:25.836651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:25.837049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:25.896367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] MFA Rules not processed for user `3051d2e633eb4f2599365c6125356663`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:25.897979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:25.991943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 837/1668] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:26.006823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:26.008588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:26.127636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df1b9e-5f0c-44f3-bf18-e4a42262eefb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:26.137281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:26.139305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.139956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.141660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.158179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:26.160447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a8bf8-d2d0-4eb0-bbb7-a05428646b38 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:26.176446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 832/1669] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:26.667952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:26.694058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ab09-4654-4695-acbd-b40c9cfc940c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:26.719323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:26.722150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.722710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.723656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.829535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:26.848508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ae610-71fc-4391-a82d-67d46cc7234f None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.849019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ae610-71fc-4391-a82d-67d46cc7234f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.849387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ae610-71fc-4391-a82d-67d46cc7234f None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:26.854671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 833/1670] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:25:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:26.900276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:26.902491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec52b3e-c582-4707-b88f-91fa824d7b7a service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:26.928158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 838/1671] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:26.987022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441625f-d971-4426-96f1-590b3c1ca72e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:26.987965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441625f-d971-4426-96f1-590b3c1ca72e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:26.988445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441625f-d971-4426-96f1-590b3c1ca72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.043170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3441625f-d971-4426-96f1-590b3c1ca72e None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:27.046590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3441625f-d971-4426-96f1-590b3c1ca72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:27.206447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 834/1672] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:27.251198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c0e1-5361-4604-ad42-7c035ba6cb40 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.251566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c0e1-5361-4604-ad42-7c035ba6cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.251892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0c0e1-5361-4604-ad42-7c035ba6cb40 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.254659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 839/1673] 10.222.0.27 () {54 vars in 961 bytes} [Mon Aug 8 02:25:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:27.269522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec0ff-783d-4afc-a440-b4cd67ed0a6c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.269810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec0ff-783d-4afc-a440-b4cd67ed0a6c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.270084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ec0ff-783d-4afc-a440-b4cd67ed0a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.330663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-416ec0ff-783d-4afc-a440-b4cd67ed0a6c None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:27.346693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-416ec0ff-783d-4afc-a440-b4cd67ed0a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:27.383701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 835/1674] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Aug 8 02:25:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:27.393831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0c0e1-5361-4604-ad42-7c035ba6cb40 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:27.427291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0c0e1-5361-4604-ad42-7c035ba6cb40 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:27.443388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:27.444023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.444304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.444579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.490331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:27.499682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117cdb9-6c7a-4ab4-a942-db542161822a service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:27.512372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:27.538216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 840/1675] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:27.541931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.542359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.542648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.624994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] MFA Rules not processed for user `5c8ae69712c14878bd2aab6d33512b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:27.626476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:27.799088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 836/1676] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:27.810146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799ee1-83a9-463b-8dbc-c3c291ad9d63 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:27.811275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799ee1-83a9-463b-8dbc-c3c291ad9d63 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:27.811275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799ee1-83a9-463b-8dbc-c3c291ad9d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:27.857221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d4799ee1-83a9-463b-8dbc-c3c291ad9d63 None None] MFA Rules not processed for user `5c8ae69712c14878bd2aab6d33512b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:27.858313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d4799ee1-83a9-463b-8dbc-c3c291ad9d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:27.894243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 841/1677] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:27.905890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:27.908252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:28.035838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7256de-fb12-407f-83b4-5c04f51945ac None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:28.053542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:28.056266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:28.056905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:28.059270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:28.059270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:28.076126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:28.082772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fde071-2b63-4569-9213-5268fdd7ebb9 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:28.117163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 837/1678] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:29.481680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cef89-fe85-4b26-8124-302b159db225 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.483950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cef89-fe85-4b26-8124-302b159db225 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.484412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cef89-fe85-4b26-8124-302b159db225 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.487010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 842/1679] 10.222.0.27 () {54 vars in 912 bytes} [Mon Aug 8 02:25:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:29.510012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.510923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.511350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.603628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:29.606236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.656929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.657201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.657633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.667374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 838/1680] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:29.703871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.705626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.720540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e23d7-7596-44e6-aee3-92a59915b876 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.734631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] MFA Rules not processed for user `d451fb2f26ce4f50b010fb2a387c4bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:29.735450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.736266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.736816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.737265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.739292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:29.759757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.761724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.782621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 839/1681] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:29.864927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 843/1682] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:29.882479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:29.916227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c1ae6-50b6-4649-846c-26f35d2c3c4e service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:29.926535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:29.927557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:29.928062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:29.928544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:29.945762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:29.948472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140f82c-9567-4f33-8bd6-a4655e41cbb2 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:29.999545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 840/1683] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:30.128168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:30.132729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:30.235641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:30.316917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2cd8e-e79b-4372-a90f-a2093bc68255 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:30.335000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:30.338134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:30.338473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:30.338693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] PATH_INFO: `/v3/users/e223926a7d054350a86ba5943908737a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:30.346424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e223926a7d054350a86ba5943908737a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:30.348385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:30.421277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:30.421790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:30.422230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:30.480399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] MFA Rules not processed for user `eb4c92edcb024e32a549650eb85f6952`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:30.492140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:30.629769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:30.670946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 841/1684] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:30.688890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:30.711433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb63e5e-3b72-4a3f-8b2f-e0e2dce5b893 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:30.726896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:30.732150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:30.732150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:30.732150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:30.735706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e223926a7d054350a86ba5943908737a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.754598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:30.754881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e223926a7d054350a86ba5943908737a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.756682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bf964-8c7f-478b-a7b5-9458f6d749ea service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:30.781135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 842/1685] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:30.796532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e223926a7d054350a86ba5943908737a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.797922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e223926a7d054350a86ba5943908737a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.800463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e223926a7d054350a86ba5943908737a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:30.973370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 844/1686] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:30 2022] DELETE /identity/v3/users/e223926a7d054350a86ba5943908737a => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:30.984743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de7-8f01-4317-b3a0-8a85f21dd583 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:30.985039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de7-8f01-4317-b3a0-8a85f21dd583 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:30.985337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de7-8f01-4317-b3a0-8a85f21dd583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:31.037462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bf000de7-8f01-4317-b3a0-8a85f21dd583 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:31.046481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bf000de7-8f01-4317-b3a0-8a85f21dd583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:31.214784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 843/1687] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:31.433567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:31.435821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:31.655925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77e6b-cc89-475c-bad5-3b923950e48b None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:31.656522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77e6b-cc89-475c-bad5-3b923950e48b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:31.656522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77e6b-cc89-475c-bad5-3b923950e48b None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:31.662706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 844/1688] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:31.699665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7fc61-98ee-43d1-a0fd-d287a1555ad8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:31.704524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:31.705417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:31.705844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:31.706527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:31.723614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:31.729641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:31.758967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 845/1689] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:31.788166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:31.792686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:31.792686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:31.834449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] MFA Rules not processed for user `46c46c81aeb14e189c21aa484404ad9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:31.843766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.084827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 845/1690] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:32.266447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:32.269367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.284365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:32.286885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:32.335363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25638-56a7-441c-a9dc-6485940501dd service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:32.348252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf4537-43ca-4f4e-aab8-fa3e22d98c09 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:32.354825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:32.357227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.358022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.358947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.364479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:32.365832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.367603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.368758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] PATH_INFO: `/v3/users/8a43738760144d429a545e9a18188b7c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.374118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a43738760144d429a545e9a18188b7c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:32.378885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:32.388941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:32.391505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:32.420896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 846/1691] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:32.817622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a43738760144d429a545e9a18188b7c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:32.840313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a43738760144d429a545e9a18188b7c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:32.886700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:32.911711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a43738760144d429a545e9a18188b7c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:32.915203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a43738760144d429a545e9a18188b7c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:32.918107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a43738760144d429a545e9a18188b7c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:32.948733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:32.967908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba50b6-e56c-4418-8558-56097a8a0cd0 service placement] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:32.992650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:32.992650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:32.992650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:32.992650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:32.994165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:33.017430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:33.019422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5a0bf-0a77-4f31-b2bf-0a12e70e3e60 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:33.041118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 847/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:33.042637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d734b-6339-4f0b-8dee-ea1d5719d515 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:33.043022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d734b-6339-4f0b-8dee-ea1d5719d515 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:33.043519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d734b-6339-4f0b-8dee-ea1d5719d515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:33.054533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 846/1693] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:32 2022] DELETE /identity/v3/users/8a43738760144d429a545e9a18188b7c => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:33.070284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-055d734b-6339-4f0b-8dee-ea1d5719d515 None None] MFA Rules not processed for user `5c8ae69712c14878bd2aab6d33512b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:33.070284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-055d734b-6339-4f0b-8dee-ea1d5719d515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:33.148083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:33.198773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 848/1694] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:33.791730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:33.794386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.084167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0946ce-1f7a-4ea5-ab70-075916edef8f admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.097825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.099321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.099607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.099884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] PATH_INFO: `/v3/projects/4b26f1294be945b1bfb2e5e0b47cc2b3` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.102009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b26f1294be945b1bfb2e5e0b47cc2b3)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.115942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.403549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b26f1294be945b1bfb2e5e0b47cc2b3', 'request_id': 'req-04b8b17b-af76-48ea-97e0-6a9bc025f888'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.578523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b26f1294be945b1bfb2e5e0b47cc2b3', 'request_id': 'req-04b8b17b-af76-48ea-97e0-6a9bc025f888'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.591705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b26f1294be945b1bfb2e5e0b47cc2b3', 'request_id': 'req-04b8b17b-af76-48ea-97e0-6a9bc025f888'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.629994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] The token cache is being invalidate because project 4b26f1294be945b1bfb2e5e0b47cc2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:34.633446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.635572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-04b8b17b-af76-48ea-97e0-6a9bc025f888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:34.645104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.645780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 847/1695] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:33 2022] DELETE /identity/v3/projects/4b26f1294be945b1bfb2e5e0b47cc2b3 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:34.646254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.646695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.669122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40040-74a2-407d-8ced-e0161d78a02c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.670116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40040-74a2-407d-8ced-e0161d78a02c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.671417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40040-74a2-407d-8ced-e0161d78a02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.714622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:34.718066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.738824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-97f40040-74a2-407d-8ced-e0161d78a02c None None] MFA Rules not processed for user `2aeb52d51d3f44319bb4e114858222f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:34.740548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-97f40040-74a2-407d-8ced-e0161d78a02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.849307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 849/1696] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:34.849307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:34.849307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.868384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 848/1697] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:34.868866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d750281-a22a-49c4-8bb7-1dbd6935450a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:34.870347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00b06-50a3-42c6-b76d-6d76e15603bb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.871255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00b06-50a3-42c6-b76d-6d76e15603bb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.871748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00b06-50a3-42c6-b76d-6d76e15603bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.889050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:34.889050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.889050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.889050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] PATH_INFO: `/v3/projects/d7025c9928ff43feb51778ce5724e0ec` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:34.890324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7025c9928ff43feb51778ce5724e0ec)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:34.892267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:34.932202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-8ee00b06-50a3-42c6-b76d-6d76e15603bb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:34.935758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8ee00b06-50a3-42c6-b76d-6d76e15603bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:34.953911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 849/1698] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:34.994075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:34.995468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:34.995906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.017484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:35.024537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] MFA Rules not processed for user `2aeb52d51d3f44319bb4e114858222f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:35.078316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.078316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 850/1699] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:35.078316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.078316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654a4fe-3db6-47c7-9709-3f73ff639ab9 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.080042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.080042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.080042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.080042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.080319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:35.080319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.080319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.095662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 851/1700] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:35.097281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.099067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.209284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7025c9928ff43feb51778ce5724e0ec', 'request_id': 'req-9239d138-1541-4e5e-9eab-c6856ac8ce4c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.315821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201415d-295b-4cb6-b5df-9d4662ca4084 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.362316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7025c9928ff43feb51778ce5724e0ec', 'request_id': 'req-9239d138-1541-4e5e-9eab-c6856ac8ce4c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.373087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7025c9928ff43feb51778ce5724e0ec', 'request_id': 'req-9239d138-1541-4e5e-9eab-c6856ac8ce4c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.412390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] The token cache is being invalidate because project d7025c9928ff43feb51778ce5724e0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:35.412545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.420383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:35.430836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 850/1701] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:34 2022] DELETE /identity/v3/projects/d7025c9928ff43feb51778ce5724e0ec => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:35.435076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.441291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.478438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.480935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.481322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.482868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.491235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.635642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239d138-1541-4e5e-9eab-c6856ac8ce4c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.642073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.643812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.644274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.644815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.645623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.649902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.711410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:35.715034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b42316-60a3-438d-af46-10f6e2bb7738 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:35.737904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 852/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:35.739252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.739543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.741668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:35.788649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] MFA Rules not processed for user `578cd837d9fe442587f8c6c8528334c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:35.791161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.795566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 851/1703] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/projects => generated 375 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:35.797843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:35.805090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:35.989393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e7126-be69-4e92-9939-47157ed59641 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:35.995389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:35.998791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:35.999271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:35.999767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] PATH_INFO: `/v3/users/07405bc5defb4eb5a2e18891a549bc50` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.002105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07405bc5defb4eb5a2e18891a549bc50)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.004255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.004921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 853/1704] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:36.010651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.038237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c92f7c-cb94-49ff-af60-97d666848690 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.055991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.057831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.058123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.058406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.062200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.064833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.240289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 854/1705] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:36.244536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.256936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcb05-2101-468c-aa77-ebaa54bf32cf admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.263279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.263982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.264273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.264620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.265863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.267697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.283418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 855/1706] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:36.284637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.307588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7242-fd24-4104-b49b-d88b47f3682c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.313558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.314185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.314620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.315090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.315773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.317948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.333401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 856/1707] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:36.335848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:36.350069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf593b8a-80cc-4c7f-8857-8f840ded5d95 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:36.355669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07405bc5defb4eb5a2e18891a549bc50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:36.368000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:36.369654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.371317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.371620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] PATH_INFO: `/v3/projects/d2f4a8558d8242c88877511889b2d6e5/users/0925fdcf05e442e3924759a24a3ee099/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.386649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07405bc5defb4eb5a2e18891a549bc50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:36.417576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07405bc5defb4eb5a2e18891a549bc50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:36.418766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07405bc5defb4eb5a2e18891a549bc50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:36.420652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c089909a-3a91-49b8-8d23-b91a882caf40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07405bc5defb4eb5a2e18891a549bc50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:36.421249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f4a8558d8242c88877511889b2d6e5, user_id=0925fdcf05e442e3924759a24a3ee099, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:36.424326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1161f5-2eaa-4761-9bcc-e783f7463caa admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:36.439110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:36.612165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 852/1708] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:35 2022] DELETE /identity/v3/users/07405bc5defb4eb5a2e18891a549bc50 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:36.613191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 857/1709] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:36 2022] PUT /identity/v3/projects/d2f4a8558d8242c88877511889b2d6e5/users/0925fdcf05e442e3924759a24a3ee099/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:36.627133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.627774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.628028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.687769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] MFA Rules not processed for user `0925fdcf05e442e3924759a24a3ee099`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:36.692526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.777697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be44260-5d51-46aa-950f-a44e88f4973a None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.777987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be44260-5d51-46aa-950f-a44e88f4973a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.778525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be44260-5d51-46aa-950f-a44e88f4973a None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.779992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 858/1710] 10.222.0.27 () {54 vars in 912 bytes} [Mon Aug 8 02:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:25:36.787538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:36.787861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:36.788148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:36.822919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:36.824171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:36.828820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 853/1711] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:36.920759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 859/1712] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:37.175371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:37.177471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:37.213665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:37.215419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:37.231252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92881e08-55ef-46c3-960d-2cd43342aa15 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:37.236999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:37.238462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.239441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.239938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.261601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:37.265033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386945e-141e-47ff-9ced-ff2dc6e0c9b3 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.302355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 860/1713] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:37.330311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61701221-5e9d-408f-a7fa-92eef162d3da None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:37.342456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:37.343112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.343390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.343666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] PATH_INFO: `/v3/projects/192d598b1b0e4ebcbb22ddbb40546624` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.354279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192d598b1b0e4ebcbb22ddbb40546624)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:37.354279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:37.613277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9278-efe5-4c2e-b518-149906e4b9eb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.613277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9278-efe5-4c2e-b518-149906e4b9eb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.613544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9278-efe5-4c2e-b518-149906e4b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.644244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-db4d9278-efe5-4c2e-b518-149906e4b9eb None None] MFA Rules not processed for user `d451fb2f26ce4f50b010fb2a387c4bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:37.645812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-db4d9278-efe5-4c2e-b518-149906e4b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:37.647664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192d598b1b0e4ebcbb22ddbb40546624', 'request_id': 'req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:37.707829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 861/1714] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:37.841962 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192d598b1b0e4ebcbb22ddbb40546624', 'request_id': 'req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:37.849340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192d598b1b0e4ebcbb22ddbb40546624', 'request_id': 'req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:37.863653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] The token cache is being invalidate because project 192d598b1b0e4ebcbb22ddbb40546624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:37.868744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:37.868744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-99cac8c5-ec3a-47ab-9cd9-5d3c7c445b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:37.868744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 854/1715] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:37 2022] DELETE /identity/v3/projects/192d598b1b0e4ebcbb22ddbb40546624 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:37.871762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:37.872655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:37.872992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:37.895515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:37.897471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.009862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.010416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.010692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.019508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 862/1716] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:38.037574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:38.038937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.053650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.061293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 855/1717] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:38.068658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9324de-5edf-4e94-87cc-d080ea86d9b8 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.074521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.075433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.075893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.076353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.077403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:38.077872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.079692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.104049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 863/1718] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:38.108897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.124803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc149b-f35e-4b64-8539-0029a6087365 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.139206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.140104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.142521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.143704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.144726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.156495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.302111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 856/1719] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:38 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:38.307072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.320576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:38.324542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37330b4c-605d-4ad5-b6e8-6c2e5ba2cc1b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.371774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.374928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.539776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.541552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.542395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.543103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.544437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.547499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.577852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c3889-9330-445b-8cb2-e22396a828ae admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.614171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.614802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.615090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.615360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.617447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:38.745373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 864/1720] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:38 2022] POST /identity/v3/users => generated 286 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:38.746643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.791514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb113e-7aa3-4d95-ad22-993945e6b897 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.808495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.812134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.813918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.814926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.817223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.818209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.820142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.826279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.841736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 857/1721] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:25:38.872370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 865/1722] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:38.878123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.891597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3897a0-7af4-418f-ba2e-03a455e63ccc service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.899822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.900871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.901352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.902255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.903099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.904971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80205637-40a6-4fad-a037-1586fb739b54 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:38.926484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 858/1723] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:38.928024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:38.940748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c321f-5da7-461a-b83d-b75d4c71b18e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:38.958278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:38.962477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:38.963418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:38.963893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] PATH_INFO: `/v3/projects/b5802203266940ac9b87f23d6b233904/users/2a375ff33f2c4e0092ab8d00aed7951d/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:38.990756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5802203266940ac9b87f23d6b233904, user_id=2a375ff33f2c4e0092ab8d00aed7951d, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:38.994366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84745d05-8cf7-45ea-8357-636d224bcdf6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.098888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:39.120786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 866/1724] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:38 2022] PUT /identity/v3/projects/b5802203266940ac9b87f23d6b233904/users/2a375ff33f2c4e0092ab8d00aed7951d/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:39.132870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.133167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.133465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.138255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9739-23c4-41a5-82c7-084feab1038c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.138738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9739-23c4-41a5-82c7-084feab1038c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.139164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9739-23c4-41a5-82c7-084feab1038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.246820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b99e9739-23c4-41a5-82c7-084feab1038c None None] MFA Rules not processed for user `eb4c92edcb024e32a549650eb85f6952`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.248881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b99e9739-23c4-41a5-82c7-084feab1038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.254431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] MFA Rules not processed for user `2a375ff33f2c4e0092ab8d00aed7951d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.259817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.402946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:39.446817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 867/1725] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:39.447522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.448761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.448761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.505954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 859/1726] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 08 02:25:39.515111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:39.668331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:39.669928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:39.718663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0748d1-bae0-4df0-baca-5fcb6f03b04b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:39.728251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:39.728925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:39.735544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:39.735544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:39.735544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:39.735544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:39.805432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 868/1727] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:39.939971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 860/1728] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/projects => generated 369 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:39.944145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:39.956891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f509-9484-41eb-93b8-3bcf222e532e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.032517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.033365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.033749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.034153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.034971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.037373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.159545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 869/1729] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:39 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:40.161528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.181060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b12b9f-c634-4e26-b5b2-34e9604ead64 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.196381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.198302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.198392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.198677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.200781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.202512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.227694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 861/1730] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:40.229860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.256830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f80ede-77d9-4e13-8f33-4fd7beda7376 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.256830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.256830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.256830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.257206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] PATH_INFO: `/v3/projects/4e554cc8aa064e7c89930ecef66846da/users/5851dde37ba94ecdaef494b793970906/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.277145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e554cc8aa064e7c89930ecef66846da, user_id=5851dde37ba94ecdaef494b793970906, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.281959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.303792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.309860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.418361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 870/1731] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:40 2022] PUT /identity/v3/projects/4e554cc8aa064e7c89930ecef66846da/users/5851dde37ba94ecdaef494b793970906/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:40.423604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.442404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d5e-24b4-42a4-a117-91f0bd954e1c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.533212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.533960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.534304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.534683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.601466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648b4c-ef6b-46d0-8756-7e410d838701 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.608776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.609446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.609787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.610250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.610720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.612631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.634885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.637320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade6646-7247-498b-83cc-c9f4c4e4499f service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.645297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 871/1732] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:40.645297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.665276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 862/1733] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:25:40.669000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e820-a0bf-4e86-b376-4260fda1d885 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:40.680414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:40.681240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.681547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.681906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] PATH_INFO: `/v3/projects/4e554cc8aa064e7c89930ecef66846da/users/5851dde37ba94ecdaef494b793970906/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.683953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e554cc8aa064e7c89930ecef66846da, user_id=5851dde37ba94ecdaef494b793970906, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:40.689570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:40.711246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 872/1734] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:40 2022] PUT /identity/v3/projects/4e554cc8aa064e7c89930ecef66846da/users/5851dde37ba94ecdaef494b793970906/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:40.715249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:40.715548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:40.722562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:40.804462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] MFA Rules not processed for user `5851dde37ba94ecdaef494b793970906`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:40.806281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:40.935530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 863/1735] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:40.948801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:40.951929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.021360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8399-ad1e-4c7b-b489-5df2b6d4af30 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.025523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.026326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.026946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.027560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.065672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.077515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4ee56-917b-4a4b-80c9-c168feed5198 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.134549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 873/1736] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:41.282476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:41.646998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:41.648763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:41.670026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48a2f-0751-4239-b8f1-3ab1f90580ad None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:41.674785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:41.675581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:41.676001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:41.676384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:41.839111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:41.839513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:41.890854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 864/1737] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:42.695038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:42.697051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:42.697426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:42.754121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:42.757374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:42.897004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:42.984072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 874/1738] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Aug 8 02:25:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:43.375760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.377387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.473882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.473882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.493972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93257cac-96e9-485e-ac80-4e0be097ae09 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.499833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.501010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.502215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.502811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.539022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.540812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.553607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 875/1739] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:43.554966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:43.563071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:43.578610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3d623-a4d1-4264-befe-84348856615a service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.601767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.602608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.603424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.605018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.624043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920fc13-f187-4dab-b4c1-4b0d015878d3 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:43.629949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.635666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad84ba3-c440-409c-a7b2-61452141333a service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.642859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:43.651176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:43.651176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:43.651176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] PATH_INFO: `/v3/users/0925fdcf05e442e3924759a24a3ee099` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:43.651176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0925fdcf05e442e3924759a24a3ee099)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:43.651176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:43.657327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 876/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:43.713800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:44.043505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0925fdcf05e442e3924759a24a3ee099'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:44.113145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0925fdcf05e442e3924759a24a3ee099'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:44.186091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0925fdcf05e442e3924759a24a3ee099'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:44.188278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0925fdcf05e442e3924759a24a3ee099'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:44.190409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0925fdcf05e442e3924759a24a3ee099'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:44.377392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 865/1741] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:43 2022] DELETE /identity/v3/users/0925fdcf05e442e3924759a24a3ee099 => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:44.384854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4a85-d8be-4267-b90d-2d101573337f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.385300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4a85-d8be-4267-b90d-2d101573337f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.385700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4a85-d8be-4267-b90d-2d101573337f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.456647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3c4f4a85-d8be-4267-b90d-2d101573337f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:44.467951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3c4f4a85-d8be-4267-b90d-2d101573337f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.660595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 877/1742] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:44.752906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:44.754902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:44.907302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb652b6-1167-47f6-b0d4-addac457967a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:44.920369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:44.921140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:44.921559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:44.921896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:44.945065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:44.960209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3bba-fcb3-46d7-b751-b87af8c910a6 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:44.974468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 866/1743] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:47.229378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.231015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.242136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.242136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 878/1744] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:25:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:47.246603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.249896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.250409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:47.265149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:47.306222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:47.307407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.540552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 867/1745] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:47.914451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:47.916864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:47.960464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8ebf0-f14e-498a-ab3a-baf4adc2bc84 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:47.978148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:47.983258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:47.984592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:47.993520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.056935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.066154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eead1-5efe-4ed9-aba3-4042efcde986 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.089954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 879/1746] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:48.138897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:48.138897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.164263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360db730-1cef-4337-b18d-4d0667b35b86 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:48.188994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:48.189229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.189405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.189859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] PATH_INFO: `/v3/projects/d2f4a8558d8242c88877511889b2d6e5` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.197648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f4a8558d8242c88877511889b2d6e5)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:48.207313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:48.298355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:48.449037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f4a8558d8242c88877511889b2d6e5', 'request_id': 'req-7947370f-3d59-485e-a4f1-f752c4c661a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.562371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f4a8558d8242c88877511889b2d6e5', 'request_id': 'req-7947370f-3d59-485e-a4f1-f752c4c661a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.575733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f4a8558d8242c88877511889b2d6e5', 'request_id': 'req-7947370f-3d59-485e-a4f1-f752c4c661a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.609961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] The token cache is being invalidate because project d2f4a8558d8242c88877511889b2d6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:48.610262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.614207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7947370f-3d59-485e-a4f1-f752c4c661a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:48.618075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 868/1747] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:48 2022] DELETE /identity/v3/projects/d2f4a8558d8242c88877511889b2d6e5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:48.627932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.631721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.631721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.685864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:48.696801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:48.905050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 880/1748] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:48.913498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:48.914452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:48.915375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:48.984411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:48.988541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:49.031221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 869/1749] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:49.035947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.052546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509487-96d5-41c5-a7eb-0c673039e42f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.072704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.075480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.075850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.077184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.078282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:49.080434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.082784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.108270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 881/1750] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:25:49.115110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.130413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33168cd3-82b0-4823-abce-7f72a8b8e18c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.136749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.137656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.138229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.138721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.139579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.142097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:49.575152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 870/1751] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:49 2022] POST /identity/v3/projects => generated 405 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:49.621825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:49.648309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe8eb-e20a-4d01-af67-e7d032f2636e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:49.898270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:49.899411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:49.899884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:49.912176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:49.914310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:49.919773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.302026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 882/1752] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:49 2022] POST /identity/v3/users => generated 303 bytes in 682 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:50.306093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.322610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5edfb1-2833-496e-bfd3-c5f1437dc02c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.346252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.347817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.348161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.348486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.349314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.364498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.426315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.430048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 871/1753] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:50.445101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085a4d-cba0-4549-8580-a7c76bee250e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.473144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.473950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.474225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.474502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.475170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.476849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8051b7-9c9a-4609-a2cf-78473ae42288 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.514426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 883/1754] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:50.515965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:50.545667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac103c-f46b-4792-a695-a48381135ba9 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:50.554310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:50.556409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.568586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.569320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] PATH_INFO: `/v3/projects/ca95a8c202574abc8df07d1590e484a6/users/3945e9b29e1a49aa8dfad5b9fc1d7c3e/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.646948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca95a8c202574abc8df07d1590e484a6, user_id=3945e9b29e1a49aa8dfad5b9fc1d7c3e, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:50.656767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f94a4-ec8a-445f-825f-c3d880fa68dc admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:50.787628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 872/1755] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:50 2022] PUT /identity/v3/projects/ca95a8c202574abc8df07d1590e484a6/users/3945e9b29e1a49aa8dfad5b9fc1d7c3e/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:50.793653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96e85-594b-457e-8eb5-056bca968fb8 None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.797086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.797745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96e85-594b-457e-8eb5-056bca968fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.798224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.798693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96e85-594b-457e-8eb5-056bca968fb8 None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.799159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.800872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 884/1756] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:25:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:25:50.863856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] MFA Rules not processed for user `3945e9b29e1a49aa8dfad5b9fc1d7c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:50.865583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:50.894881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:50.895164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:50.895627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:50.954281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:50.955814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:51.040540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.129455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 873/1757] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:51.130424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.223908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 885/1758] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:25:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:51.272561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087f1ed-f14d-406e-a99e-3b0b13011b96 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.285843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.286510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.286795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.287073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.287824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.290311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.471557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 874/1759] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:51 2022] POST /identity/v3/projects => generated 403 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:51.475474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.488262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d82303-ee54-42b8-9aae-d055d4d5933f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:51.642310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:51.717484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:51.720056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:51.720056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:51.720056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:51.720056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:51.722755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:51.971929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 886/1760] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:51 2022] POST /identity/v3/users => generated 308 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:51.976501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:51.999222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b31c-2231-45bd-a5aa-885a4518070b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.020435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.021043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.021325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.021594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.025227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.030025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.069404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 875/1761] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:52.069603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.086062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780b74-9581-4a04-aa3b-0ff4866fcaf6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.111906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.117383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.117816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.118200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] PATH_INFO: `/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b/users/82f9d8488f8243c19498591e4611885c/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.141727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.143856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:52.189752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a06a1186fb44f7b1bfb46dd3b4cf9b, user_id=82f9d8488f8243c19498591e4611885c, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.193486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.355019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776c3f9-6ce0-462a-a1e5-0de5f069c496 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.374817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 887/1762] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:52 2022] PUT /identity/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b/users/82f9d8488f8243c19498591e4611885c/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:52.379215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.527120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.529899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.530484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.530980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.542005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe0555-e661-49e6-9160-036cb42072c6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.553016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.553758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.553983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.554259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.555024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.557199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.576639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.581637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b581335-95a3-451a-9717-3f3b20f0dd84 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.592327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 888/1763] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:25:52.607050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:52.625557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a2837e-4499-422b-88d7-9ec5ebfd46ff admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:52.634031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 876/1764] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:52.641158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:52.641868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.642215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.642567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] PATH_INFO: `/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b/users/82f9d8488f8243c19498591e4611885c/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.648406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a06a1186fb44f7b1bfb46dd3b4cf9b, user_id=82f9d8488f8243c19498591e4611885c, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:52.651480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:52.693205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 889/1765] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:52 2022] PUT /identity/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b/users/82f9d8488f8243c19498591e4611885c/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:52.694859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:52.695152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:52.695425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:52.827226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] MFA Rules not processed for user `82f9d8488f8243c19498591e4611885c`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:52.828248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.106766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 877/1766] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:53.129323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:53.132641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:53.334686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86a9b-8dd0-45bf-828c-3cfa4fb8b03d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:53.346707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.348825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.349416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.349553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.363493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:53.365452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407d719-ddfa-4fd4-8faa-43edb5f16e25 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:53.378371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 890/1767] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:53 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:53.481443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:53.710328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:53.866293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca0f27-35a2-4874-a405-5cbe5a25d29b None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:53.879028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:53.880187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:53.880816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:53.881343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] PATH_INFO: `/v3/users/3945e9b29e1a49aa8dfad5b9fc1d7c3e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:53.882771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3945e9b29e1a49aa8dfad5b9fc1d7c3e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:53.885778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:54.167839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3945e9b29e1a49aa8dfad5b9fc1d7c3e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:54.189988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3945e9b29e1a49aa8dfad5b9fc1d7c3e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:54.226497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3945e9b29e1a49aa8dfad5b9fc1d7c3e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:54.229074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3945e9b29e1a49aa8dfad5b9fc1d7c3e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:54.233229 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3945e9b29e1a49aa8dfad5b9fc1d7c3e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:54.308083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:54.347563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 878/1768] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:53 2022] DELETE /identity/v3/users/3945e9b29e1a49aa8dfad5b9fc1d7c3e => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:54.351874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:54.352431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:54.352986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:54.395550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:54.398117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:54.510902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 891/1769] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:54.570098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:54.572190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:54.725505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3d8a6-3eb7-432a-a488-e74b6bd55f5b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:54.738978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:54.739709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:54.740059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:54.740405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:54.764030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:54.765797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ba746-1a2c-45a9-8a93-9954f2434493 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:54.788897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 879/1770] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:55.709824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:55.711694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:55.725810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2268ad-6216-4d96-8cd6-a3e36ccc014c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:55.732218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:55.733324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:55.733840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:55.734310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] PATH_INFO: `/v3/users/a88a4a28263d40f499c1d5f5a78265fb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:55.735919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88a4a28263d40f499c1d5f5a78265fb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:55.742822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:55.992060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88a4a28263d40f499c1d5f5a78265fb'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.002333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88a4a28263d40f499c1d5f5a78265fb'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.014982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88a4a28263d40f499c1d5f5a78265fb'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.015792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88a4a28263d40f499c1d5f5a78265fb'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.025519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88a4a28263d40f499c1d5f5a78265fb'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.123301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 892/1771] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:55 2022] DELETE /identity/v3/users/a88a4a28263d40f499c1d5f5a78265fb => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:56.123401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:56.123401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:56.123401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:56.142929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:56.144345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:56.209121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 880/1772] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:56.215839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:56.218045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:56.282729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf756b3-18c9-4da7-bec1-ed53cbf31cad admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:56.288633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:56.290053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:56.290343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:56.290784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:56.315960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:56.318091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:56.349717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 893/1773] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:25:56.434149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:56.436508 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:56.461241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883393-3bcc-4947-9fa7-7efe87928be6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:56.468567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:56.469265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:56.469710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:56.469996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] PATH_INFO: `/v3/projects/ca95a8c202574abc8df07d1590e484a6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:56.471416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca95a8c202574abc8df07d1590e484a6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:56.473790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:56.634250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca95a8c202574abc8df07d1590e484a6', 'request_id': 'req-f62326e6-5816-475f-a1d4-85e90744335e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.715362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca95a8c202574abc8df07d1590e484a6', 'request_id': 'req-f62326e6-5816-475f-a1d4-85e90744335e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.748006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca95a8c202574abc8df07d1590e484a6', 'request_id': 'req-f62326e6-5816-475f-a1d4-85e90744335e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.763097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] The token cache is being invalidate because project ca95a8c202574abc8df07d1590e484a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:56.763966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.764832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:56.767543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 881/1774] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:56 2022] DELETE /identity/v3/projects/ca95a8c202574abc8df07d1590e484a6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:56.781783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:56.799636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456983b7-3aa0-4e58-9784-5abbff8ff558 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:56.917282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:56.918183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:56.918464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:56.918734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] PATH_INFO: `/v3/users/82f9d8488f8243c19498591e4611885c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:56.921125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f9d8488f8243c19498591e4611885c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:56.923091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:57.184152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f9d8488f8243c19498591e4611885c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.192740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f9d8488f8243c19498591e4611885c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.206813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f9d8488f8243c19498591e4611885c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.207658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f9d8488f8243c19498591e4611885c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.208347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f9d8488f8243c19498591e4611885c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.212826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:57.319507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 894/1775] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:56 2022] DELETE /identity/v3/users/82f9d8488f8243c19498591e4611885c => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:57.468993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:25:57.511502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:57.512988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:57.530790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62326e6-5816-475f-a1d4-85e90744335e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:57.593564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:57.593564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:57.593564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:57.593564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] PATH_INFO: `/v3/projects/eb31af0de3e7407b88224e5d7fd78fb2` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:57.593912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb31af0de3e7407b88224e5d7fd78fb2)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:57.594957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:57.749288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb31af0de3e7407b88224e5d7fd78fb2', 'request_id': 'req-b4d13e49-c70e-4a87-9d2e-4c8d83320956'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.865624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb31af0de3e7407b88224e5d7fd78fb2', 'request_id': 'req-b4d13e49-c70e-4a87-9d2e-4c8d83320956'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.877268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb31af0de3e7407b88224e5d7fd78fb2', 'request_id': 'req-b4d13e49-c70e-4a87-9d2e-4c8d83320956'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.894720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] The token cache is being invalidate because project eb31af0de3e7407b88224e5d7fd78fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:57.895022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.896030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:57.898147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 882/1776] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:57 2022] DELETE /identity/v3/projects/eb31af0de3e7407b88224e5d7fd78fb2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:57.902406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:57.904393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:57.920633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8247e3-d20d-487c-998e-db541f90086a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:57.995355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:57.998511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:57.998511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:57.998511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] PATH_INFO: `/v3/users/3051d2e633eb4f2599365c6125356663` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:57.999057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3051d2e633eb4f2599365c6125356663)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.001190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.337255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3051d2e633eb4f2599365c6125356663'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.359196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3051d2e633eb4f2599365c6125356663'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.389470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3051d2e633eb4f2599365c6125356663'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.401184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3051d2e633eb4f2599365c6125356663'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.401923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8e13a6f7-1b68-44bc-94d9-a6307ebd5696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3051d2e633eb4f2599365c6125356663'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:58.522678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 895/1777] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:25:57 2022] DELETE /identity/v3/users/3051d2e633eb4f2599365c6125356663 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:58.609929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:58.614411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:58.665464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d13e49-c70e-4a87-9d2e-4c8d83320956 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:58.786588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:58.787297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:58.787522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:58.788019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] PATH_INFO: `/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:58.789086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a06a1186fb44f7b1bfb46dd3b4cf9b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:58.791186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:58.965343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a06a1186fb44f7b1bfb46dd3b4cf9b', 'request_id': 'req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.115132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a06a1186fb44f7b1bfb46dd3b4cf9b', 'request_id': 'req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.121403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a06a1186fb44f7b1bfb46dd3b4cf9b', 'request_id': 'req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.133972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] The token cache is being invalidate because project 41a06a1186fb44f7b1bfb46dd3b4cf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:59.134382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.135039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.140097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 883/1778] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:58 2022] DELETE /identity/v3/projects/41a06a1186fb44f7b1bfb46dd3b4cf9b => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:25:59.141647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9c267-6330-424f-bec9-a5a0981e0a4b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.142105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9c267-6330-424f-bec9-a5a0981e0a4b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.142193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9c267-6330-424f-bec9-a5a0981e0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.168713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.171804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.183236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-edf9c267-6330-424f-bec9-a5a0981e0a4b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:59.184842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-edf9c267-6330-424f-bec9-a5a0981e0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.273359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 896/1779] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:59.279844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fcc77-3bc9-4c68-aba2-27c46a5a23b4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.280617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.281310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.281720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.282053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] PATH_INFO: `/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.282599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.283045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.283421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.283567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3864a18f1e624ce3b61cd456df5fc2b4)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.285772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.307079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:59.308601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.319720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 897/1780] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:59.324791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.334167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a4476-08a9-4b04-aa84-1eb75b335b34 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.339554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.341500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.341892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.342377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.343175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:59.343696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.345656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.362256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 898/1781] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:25:59.378438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.387274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423f76-1f72-4696-9acd-10b5d9d233d6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.404073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.404739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.405124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.405366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.406004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.408012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.414958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3864a18f1e624ce3b61cd456df5fc2b4', 'request_id': 'req-bbbb1295-262c-4f28-af23-08c53092c0b3'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.521442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3864a18f1e624ce3b61cd456df5fc2b4', 'request_id': 'req-bbbb1295-262c-4f28-af23-08c53092c0b3'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.529484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 899/1782] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:59.533159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3864a18f1e624ce3b61cd456df5fc2b4', 'request_id': 'req-bbbb1295-262c-4f28-af23-08c53092c0b3'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.533361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.547963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] The token cache is being invalidate because project 3864a18f1e624ce3b61cd456df5fc2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:25:59.548566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.549217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bbbb1295-262c-4f28-af23-08c53092c0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:25:59.550753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 884/1783] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:25:59 2022] DELETE /identity/v3/projects/3864a18f1e624ce3b61cd456df5fc2b4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:25:59.556422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85288299-6210-458b-bfbd-f2f19ffe7b19 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.556737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85288299-6210-458b-bfbd-f2f19ffe7b19 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.557016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85288299-6210-458b-bfbd-f2f19ffe7b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.578879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-85288299-6210-458b-bfbd-f2f19ffe7b19 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:59.580145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-85288299-6210-458b-bfbd-f2f19ffe7b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.618306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab8d90-00af-4de4-bcf0-22db0efc4235 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.623426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.624076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.624358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.624634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.625352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.627373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.656789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 885/1784] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:25:59.664704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.664991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.665270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.686335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:25:59.687538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.695761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 886/1785] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:25:59.699958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.712401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873dd1b-9537-45e1-ac62-f93d8d68fc1d None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.732471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.733412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.733855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.734950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.735794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:25:59.736251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.741328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.762275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 887/1786] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:25:59.763162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.764998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 900/1787] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:25:59.770163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.778119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:25:59.784142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797407b-2cb2-41ed-9ca8-2e4c5dcda8f6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.814604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450002e-cb8b-4d93-ba45-595c66271c6a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.817356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.821638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.824281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.824603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.825929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.830196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.831028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.831456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.831875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.832717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.834499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.837589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.861386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 901/1788] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:59.865034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.887002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc3a94-e3c3-4f10-b7d9-83e691a6fd22 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.889242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.890138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.890421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.890695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.891348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.893136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.909517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 902/1789] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:25:59.910575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:25:59.923351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d955a-dd6c-40f7-9e39-3b7cc4322408 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:25:59.928174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:25:59.928954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:25:59.929231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:25:59.929507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] PATH_INFO: `/v3/projects/c51a5a2ba30d441a9fd37968abd6e90d/users/e496b9072c7e41e784b9c1b60d435894/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:25:59.947092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51a5a2ba30d441a9fd37968abd6e90d, user_id=e496b9072c7e41e784b9c1b60d435894, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:25:59.949904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b15b74-76f8-434c-bf6f-5a672f3d1147 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:25:59.992501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 888/1790] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:25:59.997304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.080380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 903/1791] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:25:59 2022] PUT /identity/v3/projects/c51a5a2ba30d441a9fd37968abd6e90d/users/e496b9072c7e41e784b9c1b60d435894/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:00.097776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1f811-62c9-46cb-b01b-898955d1d1b3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.121960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.122449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.122814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.156132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] MFA Rules not processed for user `e496b9072c7e41e784b9c1b60d435894`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.157386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.187400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.188021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.188550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.188861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.189706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.192202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.262067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 904/1792] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:00.263052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.276203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657aef-fb54-461d-8b5b-02282e0c11c6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.282008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.282617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.282890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.283160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.283778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.286447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.334147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 889/1793] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:25:59 2022] POST /identity/v3/users => generated 291 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.338475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.348979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd385-597b-4e2b-8a39-b6d24bed6dab admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.353808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.354415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.354699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.354976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.355623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.358122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.368527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 890/1794] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.372352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.384550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f51b8-46cb-4ada-9a77-745e6b911793 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.390670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.391501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.391938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.392268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.393092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.394685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.412356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 891/1795] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:00.413642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.428052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1ab2-1c0c-4a78-8088-7bd3460b70e4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.435999 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.436797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.437088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.437422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] PATH_INFO: `/v3/projects/9621e64f26e4447aa0cf1971927e210d/users/09a87184a0c34f8fb3485f97f98d86c3/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.454876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9621e64f26e4447aa0cf1971927e210d, user_id=09a87184a0c34f8fb3485f97f98d86c3, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.457618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3495a-bbda-4784-b23e-c71b8c101226 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.461975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 905/1796] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/projects => generated 369 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.465732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.551733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237025ac-07f9-4823-b9d4-70d3b555b5bb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.557721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.558514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.558801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.559072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.560710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.562921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.563150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 892/1797] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:00 2022] PUT /identity/v3/projects/9621e64f26e4447aa0cf1971927e210d/users/09a87184a0c34f8fb3485f97f98d86c3/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:00.573766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.574108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.574395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.610638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] MFA Rules not processed for user `09a87184a0c34f8fb3485f97f98d86c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:00.612111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:00.696993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 893/1798] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:00.697998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 906/1799] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:00.702451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.705309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.713213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887d78b-e832-43fe-9796-86fee6d6c6c2 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.781349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.781981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.782321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.782707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.783460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.785197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.786803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda4128-dc98-435a-872b-ac70ffb1e554 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.791775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.792620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.793364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.793811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.795147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.798377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 907/1800] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:00.798494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.799920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.810808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f7e5-41b6-4a9d-8870-40952df431c5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:00.816527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:00.816953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.817152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.817417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] PATH_INFO: `/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17/users/0a2a58b5299344519aa7a7be610ad1c4/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.834123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7f4523d0534669bb33e4bc0c4a6d17, user_id=0a2a58b5299344519aa7a7be610ad1c4, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:00.836817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848ce80-32b6-4b84-8764-ecc65a8641de admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:00.932513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 908/1801] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:00 2022] PUT /identity/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17/users/0a2a58b5299344519aa7a7be610ad1c4/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:00.933901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cc9d-85d9-488f-b286-7b0af006c1af None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:00.934400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cc9d-85d9-488f-b286-7b0af006c1af None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:00.934740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860cc9d-85d9-488f-b286-7b0af006c1af None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:00.935858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 894/1802] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:00.936881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:00.937097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 909/1803] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:26:00.939723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860cc9d-85d9-488f-b286-7b0af006c1af None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.007172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fef40-f133-4c14-87d0-d9aadce75b03 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.011967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.012628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.012905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.013804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.013977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860cc9d-85d9-488f-b286-7b0af006c1af None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.014459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.017193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646a7c2-1bf5-47b1-8970-cd76cfd4a3f1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.025014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.025676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.025958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.026296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.026980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.031779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8428f-7cf3-4e9a-8836-e6393e02c8dc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.031923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 895/1804] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.031923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.032605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.032605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.041955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.054776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.055967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.123603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 896/1805] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.124687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.136257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb688d44-1d4d-49ba-8130-824adbf2593a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.142136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.142913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.143090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.143843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] PATH_INFO: `/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17/users/0a2a58b5299344519aa7a7be610ad1c4/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.146363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7f4523d0534669bb33e4bc0c4a6d17, user_id=0a2a58b5299344519aa7a7be610ad1c4, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.149445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.164197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 897/1806] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17/users/0a2a58b5299344519aa7a7be610ad1c4/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.169343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 910/1807] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:00 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:01.174458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.174857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.175653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.176045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.220943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] MFA Rules not processed for user `0a2a58b5299344519aa7a7be610ad1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.222449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.258738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9a888-cc36-4f55-9470-b15819360218 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.264958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.266057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.266494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.266970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.268071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.270058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.283321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 898/1808] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:01.298577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.322061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:01.325897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 911/1809] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.330578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2abe6-4247-44df-8785-7bc864ff77b2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.339427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.339886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.340115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.341929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.341929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] PATH_INFO: `/v3/projects/089585041c4940ac95beaf3223a3d79d/users/d56df5130bf14b1b89b5f7e4a165acee/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.342748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.386727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=089585041c4940ac95beaf3223a3d79d, user_id=d56df5130bf14b1b89b5f7e4a165acee, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.392108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.439497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e56a4-63d7-4bfd-bae3-621ca508d476 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.443824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.444698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.444945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.448221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.458562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.460164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.472893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 912/1810] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:01.473798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.475255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.506101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 899/1811] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/089585041c4940ac95beaf3223a3d79d/users/d56df5130bf14b1b89b5f7e4a165acee/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.509795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.593653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb6d2b-7a74-40f1-97de-9d92e54a8dbd service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.608405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77965ba-f0c2-400a-bd27-ff2c4dd9ceb1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.614471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.615533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.616015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.616600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.617935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.619873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.632813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 900/1812] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:01.633913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.638398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.697186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.697813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.698124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.698390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.720161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.720161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.720323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d623b-afe6-4c6c-9b34-43cab489ef4e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.720323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.720323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.720323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.721165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.724862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 913/1813] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:01.725784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.738601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105e82b-812f-4b18-92a4-e84f4d67fce0 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.745875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.746800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.748180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.748701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] PATH_INFO: `/v3/projects/089585041c4940ac95beaf3223a3d79d/users/d56df5130bf14b1b89b5f7e4a165acee/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.751492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=089585041c4940ac95beaf3223a3d79d, user_id=d56df5130bf14b1b89b5f7e4a165acee, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.765251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e0d79-8bc8-47e0-8450-61f44e2c1b0f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.774934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 914/1814] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:01 2022] PUT /identity/v3/projects/089585041c4940ac95beaf3223a3d79d/users/d56df5130bf14b1b89b5f7e4a165acee/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:01.786205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.786485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.786759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.808483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.811166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:01.818194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] MFA Rules not processed for user `d56df5130bf14b1b89b5f7e4a165acee`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:01.819308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:01.821695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 901/1815] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:01.896941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 915/1816] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:01.910355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:01.918876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0e2c3-25d2-47c9-b19f-5b1f20357688 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:01.983742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:01.984632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:01.985023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:01.985366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:01.995271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:01.996720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:02.006770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 902/1817] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:03.344232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.354373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d11d49-6422-4ad7-bf3c-f4353a85761e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.360711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd56df5130bf14b1b89b5f7e4a165acee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '089585041c4940ac95beaf3223a3d79d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.361416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.363440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.363440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.363440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:03.363440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.365173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.382164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 916/1818] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:03.383403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.393068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24093c-2c12-4f1d-b7d3-e74d060ac378 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.397906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd56df5130bf14b1b89b5f7e4a165acee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '089585041c4940ac95beaf3223a3d79d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.398520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.398812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.399220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.399865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.401814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.577940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 903/1819] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:03 2022] POST /identity/v3/projects => generated 355 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:03.579019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.588896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096b090-5321-4cc3-b010-d8ee34ccd27e tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.659399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd56df5130bf14b1b89b5f7e4a165acee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '089585041c4940ac95beaf3223a3d79d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.661037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.661337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.661641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.662277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:03.663252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.665116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.694237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 917/1820] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:03.740118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:03.764451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9baf47b-3cbe-4d3e-877b-2af540edc5a5 tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:03.788707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd56df5130bf14b1b89b5f7e4a165acee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '089585041c4940ac95beaf3223a3d79d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:03.791989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:03.792557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:03.793095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] PATH_INFO: `/v3/projects/84ea70dfe7cc45158aca74c341b481fa` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:03.798142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorizing `identity:delete_project(project_id=84ea70dfe7cc45158aca74c341b481fa)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:03.802633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:03.981608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ea70dfe7cc45158aca74c341b481fa', 'request_id': 'req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:04.087045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ea70dfe7cc45158aca74c341b481fa', 'request_id': 'req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:04.094100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ea70dfe7cc45158aca74c341b481fa', 'request_id': 'req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:04.107691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] The token cache is being invalidate because project 84ea70dfe7cc45158aca74c341b481fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:04.108453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:04.109521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c77d3c8a-b7db-4056-b4c0-c22e4d32e54c tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:04.111576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 904/1821] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:03 2022] DELETE /identity/v3/projects/84ea70dfe7cc45158aca74c341b481fa => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:04.127598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:04.396216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:05.614185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:05.616238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:05.777098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ce26d-638c-4a22-be87-c8fb8917df2f tempest-VolumeQuotaClassesTest-1116042760 tempest-VolumeQuotaClassesTest-1116042760-project-admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:05.843252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:05.846580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:05.846580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:05.846580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] PATH_INFO: `/v3/users/09a87184a0c34f8fb3485f97f98d86c3` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:05.846580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a87184a0c34f8fb3485f97f98d86c3)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:05.847641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:06.090790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a87184a0c34f8fb3485f97f98d86c3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.098037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a87184a0c34f8fb3485f97f98d86c3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.110405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a87184a0c34f8fb3485f97f98d86c3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.111144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a87184a0c34f8fb3485f97f98d86c3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.111848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a87184a0c34f8fb3485f97f98d86c3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:06.269363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 918/1822] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:05 2022] DELETE /identity/v3/users/09a87184a0c34f8fb3485f97f98d86c3 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:06.274049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:06.274563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:06.274658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:06.296061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:06.296061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:06.441970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:06.499957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 905/1823] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:06.500644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:06.502285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:06.626070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fc6c0-2a8a-4a48-a76d-34c18fc6f53c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:06.632172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:06.633216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:06.633923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:06.634667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:06.647932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:06.651566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:06.661989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 919/1824] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:07.545343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.548106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:07.562312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc1c4-3eae-493b-a4d3-444f981a7766 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:07.604270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:07.605066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:07.605364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:07.605762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] PATH_INFO: `/v3/projects/9621e64f26e4447aa0cf1971927e210d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:07.618857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9621e64f26e4447aa0cf1971927e210d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:07.620802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:07.642401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:07.779918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9621e64f26e4447aa0cf1971927e210d', 'request_id': 'req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.888560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9621e64f26e4447aa0cf1971927e210d', 'request_id': 'req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.895506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9621e64f26e4447aa0cf1971927e210d', 'request_id': 'req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.913997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] The token cache is being invalidate because project 9621e64f26e4447aa0cf1971927e210d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:07.914697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.915372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:07.917215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 906/1825] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:07 2022] DELETE /identity/v3/projects/9621e64f26e4447aa0cf1971927e210d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:07.921808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:07.931919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb83388-31fc-4184-8d83-f84c10b8b1e6 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.019154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.019913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.020193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.020465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] PATH_INFO: `/v3/users/d56df5130bf14b1b89b5f7e4a165acee` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.021798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56df5130bf14b1b89b5f7e4a165acee)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.026993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.236198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56df5130bf14b1b89b5f7e4a165acee'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.243428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56df5130bf14b1b89b5f7e4a165acee'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.257371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56df5130bf14b1b89b5f7e4a165acee'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.258135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56df5130bf14b1b89b5f7e4a165acee'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.258741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d0695840-3af1-495c-9296-7cf2fcab2706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56df5130bf14b1b89b5f7e4a165acee'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.292166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.293604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.325462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85e83-1fab-4095-90f5-a6e7aeb3cac4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.330949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.331906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.332354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.332922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] PATH_INFO: `/v3/users/e496b9072c7e41e784b9c1b60d435894` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.334109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e496b9072c7e41e784b9c1b60d435894)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.336112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.372335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 920/1826] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:07 2022] DELETE /identity/v3/users/d56df5130bf14b1b89b5f7e4a165acee => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:08.558228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e496b9072c7e41e784b9c1b60d435894'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.566272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e496b9072c7e41e784b9c1b60d435894'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.577909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e496b9072c7e41e784b9c1b60d435894'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.578651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e496b9072c7e41e784b9c1b60d435894'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.579251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e496b9072c7e41e784b9c1b60d435894'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:08.739269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 907/1827] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:08 2022] DELETE /identity/v3/users/e496b9072c7e41e784b9c1b60d435894 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:08.739618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.740147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.740230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.766252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:08.767477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.828330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.830639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.847500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 921/1828] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:08.866739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:08.869003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:08.928001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6026-bab6-44ca-95e7-e48dd224c8e2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.946880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.947897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.948542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.949018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] PATH_INFO: `/v3/projects/089585041c4940ac95beaf3223a3d79d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.950959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=089585041c4940ac95beaf3223a3d79d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.953195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.953361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64949cc-69db-49ca-a16b-127a7c8d90ca None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:08.958665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:08.959589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:08.960150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:08.961060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:08.982030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:08.982030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5367316-527e-49b7-ade8-8319977e88ff service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:08.995034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 922/1829] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:09.128393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '089585041c4940ac95beaf3223a3d79d', 'request_id': 'req-78e49568-ce87-4768-903d-75a21dd767e2'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:09.264992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '089585041c4940ac95beaf3223a3d79d', 'request_id': 'req-78e49568-ce87-4768-903d-75a21dd767e2'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:09.273467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '089585041c4940ac95beaf3223a3d79d', 'request_id': 'req-78e49568-ce87-4768-903d-75a21dd767e2'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:09.305951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] The token cache is being invalidate because project 089585041c4940ac95beaf3223a3d79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:09.306747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:09.315019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-78e49568-ce87-4768-903d-75a21dd767e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:09.315019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 908/1830] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:08 2022] DELETE /identity/v3/projects/089585041c4940ac95beaf3223a3d79d => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:09.315788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.316073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.316345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.336105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:09.337347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.399959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 923/1831] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:09.408373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.408678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.418178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.473288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:09.481105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.499118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.501487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.503936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 909/1832] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:09.512200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b951e-a2d8-45f3-a563-d413705938ad None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.523804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.524915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.525796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.526241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.527086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:09.527545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.531377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.553027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 924/1833] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:09.553836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.566189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33ec8-62aa-4c7e-a282-624f649abdcc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.589034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.589796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.590421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.590769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.592492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.595871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.849237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.851213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:09.866841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71cf96-15a2-4c03-a87c-3e6934cdd4f0 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.873643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.874318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.874807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.875291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] PATH_INFO: `/v3/projects/c51a5a2ba30d441a9fd37968abd6e90d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.877230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51a5a2ba30d441a9fd37968abd6e90d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.881470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:09.888298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:09.896307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 910/1834] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/projects => generated 385 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:09.901811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:09.981790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2dd47-6555-4a4e-8dd7-f3a71dc942c2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:09.988198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:09.989081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:09.989506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:09.989922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:09.990863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:09.992975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.056454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51a5a2ba30d441a9fd37968abd6e90d', 'request_id': 'req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.089263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 911/1835] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:09 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:10.094087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.104566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a7448-64b8-4d8b-942e-a8bc09753e07 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.172937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51a5a2ba30d441a9fd37968abd6e90d', 'request_id': 'req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.175116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.175744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.176032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.176314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.177001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.178647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.179595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51a5a2ba30d441a9fd37968abd6e90d', 'request_id': 'req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.191197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 912/1836] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:10.195403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.210494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b68ede-afaa-4335-91e7-6448a925111b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.210615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] The token cache is being invalidate because project c51a5a2ba30d441a9fd37968abd6e90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:10.210615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.210615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.213949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 925/1837] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:09 2022] DELETE /identity/v3/projects/c51a5a2ba30d441a9fd37968abd6e90d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:10.214121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.214497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.214604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.214781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.215103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.215691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.218252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.219199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.234375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 913/1838] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:10.235446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.239756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.243440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e4e7e-f4a6-4f2d-8c01-0b4f8324821b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.251156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.251905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.252288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.253103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] PATH_INFO: `/v3/users/0a2a58b5299344519aa7a7be610ad1c4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.253219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34aef3-d223-4bb2-8b18-12988465c1a3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.255105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a2a58b5299344519aa7a7be610ad1c4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.257493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.258310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.260052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.260052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.260052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] PATH_INFO: `/v3/projects/dca9d10768ba4db194fbbc10cf3f5dcf/users/796df2b025fc44f598b7652346e3cc51/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.279261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca9d10768ba4db194fbbc10cf3f5dcf, user_id=796df2b025fc44f598b7652346e3cc51, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.282107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8fb7c-909c-4d71-bcdc-41fb2cdc13ab admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.409005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 914/1839] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:10 2022] PUT /identity/v3/projects/dca9d10768ba4db194fbbc10cf3f5dcf/users/796df2b025fc44f598b7652346e3cc51/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:10.438117 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.445105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.445399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.475354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] MFA Rules not processed for user `796df2b025fc44f598b7652346e3cc51`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:10.476684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.529769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a2a58b5299344519aa7a7be610ad1c4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.541672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a2a58b5299344519aa7a7be610ad1c4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.562282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a2a58b5299344519aa7a7be610ad1c4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.563335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a2a58b5299344519aa7a7be610ad1c4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.565057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a2a58b5299344519aa7a7be610ad1c4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:10.574051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 915/1840] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:10.589045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.590537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.668580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3f28f-567d-4941-b20e-4f408d828068 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:10.670234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 926/1841] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:10 2022] DELETE /identity/v3/users/0a2a58b5299344519aa7a7be610ad1c4 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:10.724171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:10.724981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:10.725261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:10.725532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:10.727445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.852867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:10.855982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:10.874479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 916/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:10.901929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:10.903579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:10.990467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3366b6-8934-4fb1-a38e-65e86363f3e9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.000224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.000937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.001273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.001549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] PATH_INFO: `/v3/users/796df2b025fc44f598b7652346e3cc51` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.003335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796df2b025fc44f598b7652346e3cc51)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.005823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.224769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796df2b025fc44f598b7652346e3cc51'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.233798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796df2b025fc44f598b7652346e3cc51'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.248784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796df2b025fc44f598b7652346e3cc51'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.250322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796df2b025fc44f598b7652346e3cc51'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.251937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4557bbd7-57ae-43ed-a1e1-c9228f4b0861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796df2b025fc44f598b7652346e3cc51'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.261526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:11.263870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.280755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c0353-d1da-4379-b0d5-c64416b276a0 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.286248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.287192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.288070 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.288550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] PATH_INFO: `/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.290085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f7f4523d0534669bb33e4bc0c4a6d17)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.292549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.454974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 927/1843] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:10 2022] DELETE /identity/v3/users/796df2b025fc44f598b7652346e3cc51 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:11.462612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.462960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.463541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.477768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f7f4523d0534669bb33e4bc0c4a6d17', 'request_id': 'req-7cba534b-fc43-40ab-b81f-705fbb87a54b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.503154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:11.505057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.577508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 928/1844] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:11.590844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:11.597570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.640640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f7f4523d0534669bb33e4bc0c4a6d17', 'request_id': 'req-7cba534b-fc43-40ab-b81f-705fbb87a54b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.650792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f7f4523d0534669bb33e4bc0c4a6d17', 'request_id': 'req-7cba534b-fc43-40ab-b81f-705fbb87a54b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.661617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957aadd1-da1e-40f6-be45-218f3c63800c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.666667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] The token cache is being invalidate because project 4f7f4523d0534669bb33e4bc0c4a6d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:11.667066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.667860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.668591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.668869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-7cba534b-fc43-40ab-b81f-705fbb87a54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:11.669270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.669559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.671579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.677784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 917/1845] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:11 2022] DELETE /identity/v3/projects/4f7f4523d0534669bb33e4bc0c4a6d17 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:11.679760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.680111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.680478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.703416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.704227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:11.705096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccedd8c-97b7-407e-b063-6ddaa0b2025d service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.705740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.717556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 929/1846] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:11.730742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 918/1847] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:11.733891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.734187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.734598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.756658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:11.757995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:11.769545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 930/1848] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:11.770919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:11.783682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199dcbc-ba8a-4b74-b08c-d355738be718 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.789535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.790454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.790734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.791005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.793886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:11.793886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.793886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.812165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 919/1849] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:11.812885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:11.824913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5551590-49f6-4df7-8891-3b8f380fca0c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:11.830518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:11.831290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:11.831632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:11.832019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:11.832873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:11.835394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:11.985114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 931/1850] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:11 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:11.989603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.002917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351806a2-bd6c-433b-9595-961b6623402e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.071948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.072626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.072946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.073221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.073908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.077558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.108841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:12.224704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 920/1851] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:11 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:12.238006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.242113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2a284-406f-4c5f-9a7a-f5ccd2874b24 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.247294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.248299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.248867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.249289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.250110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.251854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.265639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 932/1852] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:12.269894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.282149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e58-9c22-4db0-b4ab-f8f0d3a6ab0c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.289366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.289996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.290279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.290553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.291285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.292961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.308122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 921/1853] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:12.312606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.313228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.316265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:12.323080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352bcb-1600-4111-aa8d-b16ea6e586c1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.328728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.329397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.329732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.330157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] PATH_INFO: `/v3/projects/7d15f908924d4461b49937052f21707e/users/b30fa691dd9544d0b8ef6c0b7b83913d/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.347498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a743ee2-536d-46fe-ba30-b90a89e4d8bd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.373423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.374440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.374792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.375322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] PATH_INFO: `/v3/projects/dca9d10768ba4db194fbbc10cf3f5dcf` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.377187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca9d10768ba4db194fbbc10cf3f5dcf)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.378182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d15f908924d4461b49937052f21707e, user_id=b30fa691dd9544d0b8ef6c0b7b83913d, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.379487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.383102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203b3e3-6025-4522-8bb1-495c580e50b3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.494172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 922/1854] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:12 2022] PUT /identity/v3/projects/7d15f908924d4461b49937052f21707e/users/b30fa691dd9544d0b8ef6c0b7b83913d/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:12.507963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.508455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.508835 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.512931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca9d10768ba4db194fbbc10cf3f5dcf', 'request_id': 'req-c5003598-3b17-4dfc-8e7a-e70beea4aec0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:12.545831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] MFA Rules not processed for user `b30fa691dd9544d0b8ef6c0b7b83913d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:12.545831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:12.614845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 923/1855] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:12.619938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca9d10768ba4db194fbbc10cf3f5dcf', 'request_id': 'req-c5003598-3b17-4dfc-8e7a-e70beea4aec0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:12.620089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.626627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca9d10768ba4db194fbbc10cf3f5dcf', 'request_id': 'req-c5003598-3b17-4dfc-8e7a-e70beea4aec0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:12.651989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] The token cache is being invalidate because project dca9d10768ba4db194fbbc10cf3f5dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:12.652308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:12.654484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c5003598-3b17-4dfc-8e7a-e70beea4aec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:12.655379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 933/1856] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:12 2022] DELETE /identity/v3/projects/dca9d10768ba4db194fbbc10cf3f5dcf => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:12.661016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61d4b1-7662-444c-b7a7-10a66edd7b98 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.661380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61d4b1-7662-444c-b7a7-10a66edd7b98 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.661562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61d4b1-7662-444c-b7a7-10a66edd7b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.692886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-ea61d4b1-7662-444c-b7a7-10a66edd7b98 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:12.694799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ea61d4b1-7662-444c-b7a7-10a66edd7b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:12.724254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d31182-8ee0-4d90-a6f0-4357c20e1ebb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.732734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.733521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.733810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.734086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.734889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.738266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.766894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:12.769938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 934/1857] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:12.779423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.779423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.779664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.803741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:12.803741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:12.814899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 935/1858] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:12.819388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.829493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dde9c2-51bc-4185-a0c7-760d5f04e9a0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.834153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.834765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.835495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.835495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.835981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:12.836237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.837906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.852506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 936/1859] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:12.858602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.870002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6b6a-0006-4274-9e01-3ab97ec3e5e4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.877011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.877731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.879428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.879428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.879428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.883210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:12.894093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 924/1860] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:12.898038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:12.900168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:12.971736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683ba7d-0d43-4ca4-81dc-64bd36a91ee8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:12.976486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:12.977134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:12.977420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:12.977695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:12.980435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:12.980435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.066555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 937/1861] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/projects => generated 377 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:13.069483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.099621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 925/1862] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:12 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:13.110549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.120681 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745b20-bcba-445e-b352-d34ce0c12548 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.131820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb5e24-bc10-478f-8465-29ad3441033c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.136647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.137293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.141051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.141051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.141051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.141324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.178304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.178911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.179187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.179457 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.180130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.181866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.192412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 926/1863] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:13.196443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.206647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d4074-320c-417e-9a98-1f445eb0b210 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.211380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.212010 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.212291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.212568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] PATH_INFO: `/v3/projects/07935f77f9f74029bd4449aeaf751baf/users/2aa8f4e1d6724c45b222b1f90cccd455/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.226793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07935f77f9f74029bd4449aeaf751baf, user_id=2aa8f4e1d6724c45b222b1f90cccd455, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.229513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.251832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 938/1864] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:13 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:13.255982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.265758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73e305-cc12-4543-bda5-3c0e5171431c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.270356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.270931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.271210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.271480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.272172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.273804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.284335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 939/1865] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:13.288151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.297878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f77d5-f9a2-4452-b3cc-7196012ba01c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.302442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.303009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.303291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.303565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.304179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.305739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.316759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 927/1866] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:13 2022] PUT /identity/v3/projects/07935f77f9f74029bd4449aeaf751baf/users/2aa8f4e1d6724c45b222b1f90cccd455/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:13.319361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 940/1867] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:13.320619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.323397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.334519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a73216-44e8-46b3-937a-fa92242f8e00 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.346225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:13.386177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563a4c1-c54b-4e89-af9c-7f6dbeafa483 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.390651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.391321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.391610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.391881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.392531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.394263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.399017 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.399990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.400284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.400561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] PATH_INFO: `/v3/projects/fc704bcba15148bab9aeb3e422d5800e/users/7b9dea6290eb4849b04f7cd38c009bfa/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.419908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 928/1868] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:13.422738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.451918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076712e-7e9c-4631-a87d-207c1caaa121 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.457288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc704bcba15148bab9aeb3e422d5800e, user_id=7b9dea6290eb4849b04f7cd38c009bfa, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.468015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab01ea-4a14-4f77-bf7c-e590e8caba2d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.470288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.470981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.472940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.473591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] PATH_INFO: `/v3/projects/07935f77f9f74029bd4449aeaf751baf/users/2aa8f4e1d6724c45b222b1f90cccd455/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.475734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07935f77f9f74029bd4449aeaf751baf, user_id=2aa8f4e1d6724c45b222b1f90cccd455, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.478794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38352d-ab63-4274-9b65-28a8e873d401 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.494784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 929/1869] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:13 2022] PUT /identity/v3/projects/07935f77f9f74029bd4449aeaf751baf/users/2aa8f4e1d6724c45b222b1f90cccd455/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:13.506579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.506964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.507318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.540796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] MFA Rules not processed for user `2aa8f4e1d6724c45b222b1f90cccd455`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:13.542265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.633215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 941/1870] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:13 2022] PUT /identity/v3/projects/fc704bcba15148bab9aeb3e422d5800e/users/7b9dea6290eb4849b04f7cd38c009bfa/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:13.647798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.648094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.648374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.652749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 930/1871] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:13.668361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.670680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.719155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] MFA Rules not processed for user `7b9dea6290eb4849b04f7cd38c009bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:13.720621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.762675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d32ec5-c528-474e-858f-11f6ebd9092e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.802822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 942/1872] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:13.812341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:13.813899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:13.817615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.818315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.818663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.819128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.826880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34075-d678-49a1-ac10-07ba19a50bae None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:13.838005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:13.838675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:13.838959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:13.839352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:13.852455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.854281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.880537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 943/1873] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:13.906832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:13.908467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff89bc-614c-4882-a205-328b0e7808c0 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:13.924682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 931/1874] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:13.963496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.076261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea89a2-ad50-4eaf-af1a-c210d2062af1 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.089784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.091173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.091479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.091761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] PATH_INFO: `/v3/users/b30fa691dd9544d0b8ef6c0b7b83913d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.104705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30fa691dd9544d0b8ef6c0b7b83913d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.104705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.370023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30fa691dd9544d0b8ef6c0b7b83913d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.384275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30fa691dd9544d0b8ef6c0b7b83913d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.408234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30fa691dd9544d0b8ef6c0b7b83913d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.408453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30fa691dd9544d0b8ef6c0b7b83913d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.409536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30fa691dd9544d0b8ef6c0b7b83913d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:14.542516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 944/1875] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:13 2022] DELETE /identity/v3/users/b30fa691dd9544d0b8ef6c0b7b83913d => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:14.548829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.549120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.549401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.569144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:14.570478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.637084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 932/1876] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:14.647375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:14.648936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:14.731356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ed79-0adb-4976-b007-2c0d8caa27a7 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:14.736974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:14.737740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:14.738098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:14.738447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:14.750819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:14.752963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:14.769245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 945/1877] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:15.722895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:15.724524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:15.739283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895d5de-8868-4368-a5ff-5922d39db0a6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:15.747106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:15.749264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:15.749574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:15.749879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] PATH_INFO: `/v3/projects/7d15f908924d4461b49937052f21707e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:15.751993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d15f908924d4461b49937052f21707e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:15.756090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:15.808345 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:15.923460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d15f908924d4461b49937052f21707e', 'request_id': 'req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.044926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d15f908924d4461b49937052f21707e', 'request_id': 'req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.052932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d15f908924d4461b49937052f21707e', 'request_id': 'req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.104766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] The token cache is being invalidate because project 7d15f908924d4461b49937052f21707e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:16.105313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.106206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.113141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 933/1878] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:15 2022] DELETE /identity/v3/projects/7d15f908924d4461b49937052f21707e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:16.114514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:16.136737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01161638-3a16-4ef7-b4eb-f7f2beb0878a service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:16.204123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:16.204698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:16.204984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:16.205284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] PATH_INFO: `/v3/users/2aa8f4e1d6724c45b222b1f90cccd455` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:16.207353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa8f4e1d6724c45b222b1f90cccd455)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:16.209931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:16.333405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:16.404958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa8f4e1d6724c45b222b1f90cccd455'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.413151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa8f4e1d6724c45b222b1f90cccd455'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.427702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa8f4e1d6724c45b222b1f90cccd455'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.428568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa8f4e1d6724c45b222b1f90cccd455'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.429226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98836451-1f2f-46bc-9fcf-cefa35eff876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa8f4e1d6724c45b222b1f90cccd455'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:16.573270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 946/1879] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:16 2022] DELETE /identity/v3/users/2aa8f4e1d6724c45b222b1f90cccd455 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:17.105720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.107965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:17.128819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c59b-f117-40ad-b72d-a1b7f5f151cb admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:17.221625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:17.222195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.222474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.222790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] PATH_INFO: `/v3/projects/07935f77f9f74029bd4449aeaf751baf` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.224187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07935f77f9f74029bd4449aeaf751baf)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:17.226205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:17.353676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07935f77f9f74029bd4449aeaf751baf', 'request_id': 'req-18e766f6-13f3-4566-acb6-a1985ecc9525'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:17.475742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07935f77f9f74029bd4449aeaf751baf', 'request_id': 'req-18e766f6-13f3-4566-acb6-a1985ecc9525'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:17.481785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07935f77f9f74029bd4449aeaf751baf', 'request_id': 'req-18e766f6-13f3-4566-acb6-a1985ecc9525'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:17.492751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] The token cache is being invalidate because project 07935f77f9f74029bd4449aeaf751baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:17.493038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:17.493665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-18e766f6-13f3-4566-acb6-a1985ecc9525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:17.495144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 934/1880] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:17 2022] DELETE /identity/v3/projects/07935f77f9f74029bd4449aeaf751baf => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:17.500732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.501186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.501269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.520606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:17.521813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:17.584012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 947/1881] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:17.591752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.592008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.592286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.612350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:17.613566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:17.621420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 935/1882] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:17.627058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.637331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faf852-5fb4-45d2-b5c4-606e4d3520d7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:17.642814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:17.643441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.643723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.643993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.646865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:17.646865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:17.646865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:17.658355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 948/1883] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:17.662555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.671723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb54f93-52e3-415e-9b54-25bd7cdadffb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:17.676333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:17.676963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.677247 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.677520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.678313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:17.680094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:17.786591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 936/1884] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:17 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:17.790428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.803369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99e135-c518-471d-a6a0-22e79fcd8276 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:17.859344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:17.859961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.860243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.860522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.861236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:17.863191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:17.960911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 949/1885] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:17 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:17.965097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:17.976170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b4b8-458e-4edc-bccf-cc81ba83ddab admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:17.980708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:17.981301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:17.981577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:17.982903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:17.983546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:17.985154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:17.995890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 937/1886] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:17.999954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.010931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884261-6250-4c21-80c2-4891fd19d88d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.016949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.017566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.017843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.018187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.018755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.020408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddea694-f798-499e-b1c9-3543cfdb2c5a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.032408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 950/1887] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:18.039197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.049806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b2849-2a30-4eb2-9d70-4cef4b13b49a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.054418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.055062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.055337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.055607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] PATH_INFO: `/v3/projects/922c9d86ed2f4da98644b02955e277c4/users/5abe661f776541f0a39c9af36752ea50/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.070432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=922c9d86ed2f4da98644b02955e277c4, user_id=5abe661f776541f0a39c9af36752ea50, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.073158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.173248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 938/1888] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:18 2022] PUT /identity/v3/projects/922c9d86ed2f4da98644b02955e277c4/users/5abe661f776541f0a39c9af36752ea50/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:18.184590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64ace-d732-4900-afba-b750576daf6e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.185430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64ace-d732-4900-afba-b750576daf6e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.185706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64ace-d732-4900-afba-b750576daf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.214289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bea64ace-d732-4900-afba-b750576daf6e None None] MFA Rules not processed for user `5abe661f776541f0a39c9af36752ea50`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.215490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bea64ace-d732-4900-afba-b750576daf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:18.241433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.286014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 951/1889] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:18.289499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.351332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2213236-4e65-458e-8ee9-dc5812923aac admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.355901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.356517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.356945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.358801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.358801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.359767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.477265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 939/1890] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:18.480907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64ace-d732-4900-afba-b750576daf6e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.492335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64ace-d732-4900-afba-b750576daf6e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.542382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:18.548006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.548630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.548944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.550665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.550665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.551813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.647375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 952/1891] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:18.651292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.661639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b337-1ef0-47fd-ad46-cfd169285255 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.666162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.666757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.667038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.667312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.668127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.670143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.680640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 940/1892] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:18.684643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.696060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f86f3-ea33-450a-b7d8-cd0b447e71ff admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.700710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.701385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.701662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.701932 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] PATH_INFO: `/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1/users/a3b08844fb6c42188041d0885543c8bc/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.716157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3327cdd5b7e94777a8df44f21b8abaf1, user_id=a3b08844fb6c42188041d0885543c8bc, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.718977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.801959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 953/1893] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:18 2022] PUT /identity/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1/users/a3b08844fb6c42188041d0885543c8bc/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:18.805818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.816954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e0ecb-406f-45a5-8205-8a6789057244 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.873149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.873784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.874065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.874336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.874973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.876552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86079c1-ad32-4d21-bbd2-e86cc236c94f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.886363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 941/1894] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:18.890687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:18.901185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547202c-13aa-4fb8-a2b5-d088a55d12bf admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:18.906301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:18.906951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.907232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.907532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] PATH_INFO: `/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1/users/a3b08844fb6c42188041d0885543c8bc/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.909544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3327cdd5b7e94777a8df44f21b8abaf1, user_id=a3b08844fb6c42188041d0885543c8bc, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:18.912173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:18.923176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 954/1895] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:18 2022] PUT /identity/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1/users/a3b08844fb6c42188041d0885543c8bc/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:18.933297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:18.933586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:18.933866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:18.962250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] MFA Rules not processed for user `a3b08844fb6c42188041d0885543c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:18.963467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.031687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 942/1896] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:19.041651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:19.043354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:19.108459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e323b-8861-4259-9f4c-6423574d3ad1 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:19.113790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:19.114428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:19.114710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:19.115124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:19.126395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:19.127929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7d169-5314-41bf-9a52-de9ad4122081 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:19.143061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 955/1897] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:19.727096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.068979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.369477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.433184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90057c84-69fc-46f5-8ecf-a3bc3eab16b9 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.438176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.438752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.439035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.439315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] PATH_INFO: `/v3/users/5abe661f776541f0a39c9af36752ea50` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.440503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5abe661f776541f0a39c9af36752ea50)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.442529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.655378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5abe661f776541f0a39c9af36752ea50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:20.662084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5abe661f776541f0a39c9af36752ea50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:20.673955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5abe661f776541f0a39c9af36752ea50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:20.674625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5abe661f776541f0a39c9af36752ea50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:20.675223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5abe661f776541f0a39c9af36752ea50'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:20.767079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 943/1898] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:20 2022] DELETE /identity/v3/users/5abe661f776541f0a39c9af36752ea50 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:20.768543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:20.771813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.772101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.772376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.791775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:20.793005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:20.854272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 956/1899] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:20.864355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:20.865818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:20.923364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12988e3d-f320-4563-9cb6-e2fcd8abdeef admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:20.926961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:20.927567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:20.927864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:20.928412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:20.938136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:20.939566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:20.948802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 944/1900] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:21.424317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:21.425816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:21.442631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e555db8-e9b4-405d-b814-e223c1bdb9f4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:21.448657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:21.449464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:21.449661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:21.449933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] PATH_INFO: `/v3/projects/922c9d86ed2f4da98644b02955e277c4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:21.452437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922c9d86ed2f4da98644b02955e277c4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:21.454287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:21.507802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:21.606286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922c9d86ed2f4da98644b02955e277c4', 'request_id': 'req-c02df45c-fbe3-4551-95c8-734cde772945'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:21.709730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922c9d86ed2f4da98644b02955e277c4', 'request_id': 'req-c02df45c-fbe3-4551-95c8-734cde772945'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:21.715641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922c9d86ed2f4da98644b02955e277c4', 'request_id': 'req-c02df45c-fbe3-4551-95c8-734cde772945'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:21.727044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] The token cache is being invalidate because project 922c9d86ed2f4da98644b02955e277c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:21.727336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:21.728014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:21.729579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 957/1901] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:21 2022] DELETE /identity/v3/projects/922c9d86ed2f4da98644b02955e277c4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:21.734099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:21.743622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b300a-055e-4d9e-9990-896fce99a273 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:21.770823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:21.804597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:21.805369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:21.805652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:21.805930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] PATH_INFO: `/v3/users/a3b08844fb6c42188041d0885543c8bc` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:21.807060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b08844fb6c42188041d0885543c8bc)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:21.809045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:22.023628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b08844fb6c42188041d0885543c8bc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:22.030798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b08844fb6c42188041d0885543c8bc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:22.042980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b08844fb6c42188041d0885543c8bc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:22.043699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b08844fb6c42188041d0885543c8bc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:22.044963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bccb7863-3857-429a-b2b1-7d0492cab33b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b08844fb6c42188041d0885543c8bc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:22.127337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 945/1902] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:21 2022] DELETE /identity/v3/users/a3b08844fb6c42188041d0885543c8bc => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:22.655310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:22.656742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:22.670894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df45c-fbe3-4551-95c8-734cde772945 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:22.731793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:22.732437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:22.732752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:22.733031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] PATH_INFO: `/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:22.734159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3327cdd5b7e94777a8df44f21b8abaf1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:22.736095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:22.939131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3327cdd5b7e94777a8df44f21b8abaf1', 'request_id': 'req-98dad6ca-de55-4bdf-9633-f9fdcc99e846'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:23.058471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3327cdd5b7e94777a8df44f21b8abaf1', 'request_id': 'req-98dad6ca-de55-4bdf-9633-f9fdcc99e846'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:23.065529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3327cdd5b7e94777a8df44f21b8abaf1', 'request_id': 'req-98dad6ca-de55-4bdf-9633-f9fdcc99e846'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:23.076654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] The token cache is being invalidate because project 3327cdd5b7e94777a8df44f21b8abaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:23.076974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:23.077640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-98dad6ca-de55-4bdf-9633-f9fdcc99e846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:23.079115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 958/1903] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:22 2022] DELETE /identity/v3/projects/3327cdd5b7e94777a8df44f21b8abaf1 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:23.083896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.084359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.084446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.103394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:23.104646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:23.170022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 946/1904] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:23.174154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.174441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.174712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.195089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:23.196285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:23.204308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 959/1905] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:23.208845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.218469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb3f3b-905e-49bb-820b-3643f1c70dfa None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.223043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.223640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.223917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.224187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.224873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:23.225178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.226792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.238585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 947/1906] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:23.243053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.250893 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:23.254584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ea2b5-8314-4c5a-95d3-69ef0ed87b46 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.259304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.259934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.260212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.260483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.261189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.263272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.363693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 960/1907] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:23 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:23.368039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.378492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dff0a6-ecbd-4bab-afa4-98ee41a61f01 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.434549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.435292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.435575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.435852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.436787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.438696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.470038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:23.570179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 948/1908] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:23 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:23.571432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.581694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c542dd1-046c-4ded-9e38-d5e4ac4fbf63 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.590813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.591914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.592205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.592472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.593154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.594748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.608357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 961/1909] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:23.609751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.620227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b4ac7-2370-4e1f-885d-266dc8a87308 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.630576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.630576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.630576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.630576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.630929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.630929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f327ba-6307-4ae6-813b-1871c7e1a3d7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.641007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 949/1910] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:23.645116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.655273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9b6ef-00b3-4185-93c7-556659a3d716 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.659787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.660434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.660747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.661028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] PATH_INFO: `/v3/projects/9a70fe5946fa403486635660d569a633/users/b2f8f5a03cc94382ae624f4b9afc5c69/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.676589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a70fe5946fa403486635660d569a633, user_id=b2f8f5a03cc94382ae624f4b9afc5c69, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.679457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.784288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 962/1911] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:23 2022] PUT /identity/v3/projects/9a70fe5946fa403486635660d569a633/users/b2f8f5a03cc94382ae624f4b9afc5c69/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:23.796001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.799272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.799272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.824505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] MFA Rules not processed for user `b2f8f5a03cc94382ae624f4b9afc5c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:23.826320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:23.889796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 950/1912] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:23.900849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:23.902617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:23.959902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150088-db6a-442a-91d0-ff1c77422d5a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:23.963565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:23.964172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:23.964449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:23.964756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:23.973700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:23.975208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565b582-e018-4f83-8f4d-51a4bfc7ee32 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:23.983872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 963/1913] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:25.241468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:25.424093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:25.842280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:26.550188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:27.229859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:27.378596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:28.481561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:28.721626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:29.382405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:29.578516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:30.577151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:30.887976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:31.345476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:32.714873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:32.779308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0679931-95f7-48a8-b0f2-4bfb538d6b91 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:32.784035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:32.784778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:32.785057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:32.785359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] PATH_INFO: `/v3/users/b2f8f5a03cc94382ae624f4b9afc5c69` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:32.786540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f8f5a03cc94382ae624f4b9afc5c69)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:32.788452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:32.992358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f8f5a03cc94382ae624f4b9afc5c69'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:32.999181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f8f5a03cc94382ae624f4b9afc5c69'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.011106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f8f5a03cc94382ae624f4b9afc5c69'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.011948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f8f5a03cc94382ae624f4b9afc5c69'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.012792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f8f5a03cc94382ae624f4b9afc5c69'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:33.113768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 951/1914] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:32 2022] DELETE /identity/v3/users/b2f8f5a03cc94382ae624f4b9afc5c69 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:33.118400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.118865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.118954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.139446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:33.140634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:33.200101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 964/1915] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:33.210583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:33.212095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:33.280731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7605f-ddbe-4a20-9880-d085daaf0439 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:33.284494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:33.285129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:33.285407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:33.285676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:33.295127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:33.296542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d66fd-b9c1-459f-94ec-8cb221a60102 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:33.306204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 952/1916] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:33.384197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:34.020812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.022402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.049746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c76b42-dea2-4cbf-a416-e28ebc1b54f3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.054668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.055308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.055596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.055856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] PATH_INFO: `/v3/projects/9a70fe5946fa403486635660d569a633` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.057002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a70fe5946fa403486635660d569a633)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.058943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.073137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:34.213651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a70fe5946fa403486635660d569a633', 'request_id': 'req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.315196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a70fe5946fa403486635660d569a633', 'request_id': 'req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.321086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a70fe5946fa403486635660d569a633', 'request_id': 'req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.332967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] The token cache is being invalidate because project 9a70fe5946fa403486635660d569a633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:34.333288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.333954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e2a6bd14-aec9-48ba-b4ae-f4d2252a0809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:34.335451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 965/1917] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:34 2022] DELETE /identity/v3/projects/9a70fe5946fa403486635660d569a633 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:34.340383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.340898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.340991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.361114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:34.362317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.419725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 953/1918] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:34.427790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.428113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.428396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.449506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:34.450994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:34.459126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 966/1919] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:34.463464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.473292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe98e0-2634-4d80-9328-9001ffc8b6b5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.477855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:34.481205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.481600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.493939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 954/1920] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:26:34.498966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.508310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870cea-a099-4d32-9f17-15c36168f4bf None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.513106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.513792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.514156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.514433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.515070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.517020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.703730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 967/1921] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:34 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:34.706420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.716906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318853ba-279c-4375-970b-e89d1e098227 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.774274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.774901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.775182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.775454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.776100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.778424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.875947 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 955/1922] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:34 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:34.880384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.891245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93281862-25a7-4efd-a072-fcf3355ff852 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.895765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.896399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.896706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.896981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.899030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.899226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.909484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 968/1923] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:34.913603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.923627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cced4-9a11-467b-b652-5b64f3d5f0ac admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.931012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.931012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.931012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.931012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.931382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.931653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3164e-8d5d-47ad-a218-0ed9102ec5a6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:34.942897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 956/1924] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:34.946879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:34.957356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c643c4-0e3d-4269-ab65-1b0a8bfbe55c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:34.963903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:34.964589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:34.976550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:34.976550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] PATH_INFO: `/v3/projects/69544a83ae2b4109b72f7621d8432d02/users/c5f064c1e10e4fa58dbba823949f27fc/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:34.996216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69544a83ae2b4109b72f7621d8432d02, user_id=c5f064c1e10e4fa58dbba823949f27fc, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:34.999737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.165708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 969/1925] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:34 2022] PUT /identity/v3/projects/69544a83ae2b4109b72f7621d8432d02/users/c5f064c1e10e4fa58dbba823949f27fc/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:35.178571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.178882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.179162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.212292 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] MFA Rules not processed for user `c5f064c1e10e4fa58dbba823949f27fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:35.213552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:35.276631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 957/1926] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:35.281320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.343038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1719f-e415-4a97-818f-656c6334fc18 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.347786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.348390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.348691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.348972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.349602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.351499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.492735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 970/1927] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:35.496707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.507829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098271d-976c-4d61-b0c2-fcd33b5093eb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.569422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.569422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.569422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.569422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.569771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.571219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.762255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 958/1928] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:35 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:35.766787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.866882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22fb0b-e4bf-40bc-b37d-0d64571ca345 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.866882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.866882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.866882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.867743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.867743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.867743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.867743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 971/1929] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:35.868013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.878308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba69df-5ad0-47d2-9870-792ac845277e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.884481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.885212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.885561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.886085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.886933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.889281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f02d6a-ae88-472a-8346-e977412ef62d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:35.901801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 959/1930] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:35.906994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:35.918875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6225-def5-42e2-a595-8bb9b415dddc admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:35.925193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:35.925958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:35.926334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:35.926736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] PATH_INFO: `/v3/projects/f508881a9f1c4f8d94e5edf403c23f6e/users/c08427fdc1974bc6b87a030cd9ee4423/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:35.944966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f508881a9f1c4f8d94e5edf403c23f6e, user_id=c08427fdc1974bc6b87a030cd9ee4423, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:35.947754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.094423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 972/1931] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:35 2022] PUT /identity/v3/projects/f508881a9f1c4f8d94e5edf403c23f6e/users/c08427fdc1974bc6b87a030cd9ee4423/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:36.104595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.104908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.105180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.134348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] MFA Rules not processed for user `c08427fdc1974bc6b87a030cd9ee4423`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:36.135557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:36.231917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 960/1932] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:36.238544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.307003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:36.335756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f00a7-9606-4182-9f4a-43107f6ca4dc admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.342937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.348158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.348158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.348158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.348158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.348158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.483679 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 973/1933] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:36 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:36.485746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.496423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af8e82-99cd-4d70-9455-3bd1a0d29255 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.551055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.551713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.552003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.552278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.552936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.554833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.582722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:36.685901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 961/1934] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:36 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:36.690200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.700369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d117-6425-40fd-9dea-c448630e8ff3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.705822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.706451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.706729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.707000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.707635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.709303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.721070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 974/1935] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:36.730698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.735411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b532b6-63e0-48f2-9750-116299c7983f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.739945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.740583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.740889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.741163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] PATH_INFO: `/v3/projects/8e89674d52c54167902f77d76fd2c28a/users/c608d51158854c4db1e718c4161f60fc/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.755940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e89674d52c54167902f77d76fd2c28a, user_id=c608d51158854c4db1e718c4161f60fc, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.758651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.851296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 962/1936] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:36 2022] PUT /identity/v3/projects/8e89674d52c54167902f77d76fd2c28a/users/c608d51158854c4db1e718c4161f60fc/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:36.855547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.866302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576f3a0-2843-4242-bbe3-050a7106d640 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.924368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.925021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.925307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.925585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.926235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.927828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be447d-1bd7-4a62-808e-bd8e06f326ea admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.938700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 975/1937] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:36.942633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:36.952883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ca1d-3f46-480f-b065-19dd8410c1c6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:36.957731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:36.958493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.958771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.959046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] PATH_INFO: `/v3/projects/8e89674d52c54167902f77d76fd2c28a/users/c608d51158854c4db1e718c4161f60fc/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:36.961035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e89674d52c54167902f77d76fd2c28a, user_id=c608d51158854c4db1e718c4161f60fc, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:36.963814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:36.974683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 963/1938] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:36 2022] PUT /identity/v3/projects/8e89674d52c54167902f77d76fd2c28a/users/c608d51158854c4db1e718c4161f60fc/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:36.986569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:36.986569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:36.986569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:37.015144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] MFA Rules not processed for user `c608d51158854c4db1e718c4161f60fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:37.016392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:37.101068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 976/1939] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:37.117676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:37.119531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:37.179562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf25aba-28b7-42ed-80af-2a930964ebc3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:37.183176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:37.183780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:37.184064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:37.184335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:37.196311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:37.243051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:37.244418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:37.253987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 964/1940] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:37.978507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:38.704142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:40.179718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:40.518445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:40.527899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac36ff-1a69-4482-8255-ac8df5e81e7c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:40.532030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:40.532639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:40.533001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:40.533271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:40.616432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:40.617945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:40.628509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 977/1941] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:41.121253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:41.703254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:45.292274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:45.301949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0718-7947-4143-86b9-0aa0ba507b9d service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:45.306013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:45.306991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:45.307368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:45.307704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:45.318389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:45.319883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39763b2-de77-4601-8bef-594b50d359a8 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:45.330055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 965/1942] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:47.759348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:47.761240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:47.849351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69092b-dfb6-4301-8e79-82a6520b1920 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:47.855104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:47.856145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:47.856501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:47.856799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] PATH_INFO: `/v3/users/7b9dea6290eb4849b04f7cd38c009bfa` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:47.857979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9dea6290eb4849b04f7cd38c009bfa)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:47.860156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.102591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9dea6290eb4849b04f7cd38c009bfa'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.110599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9dea6290eb4849b04f7cd38c009bfa'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.129129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9dea6290eb4849b04f7cd38c009bfa'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.130290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9dea6290eb4849b04f7cd38c009bfa'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.131368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9dea6290eb4849b04f7cd38c009bfa'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:48.249403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:48.254384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 978/1943] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:47 2022] DELETE /identity/v3/users/7b9dea6290eb4849b04f7cd38c009bfa => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:48.258910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.259362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.259450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.279913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:48.279913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.333139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:48.341818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 966/1944] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:48.351632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.353289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.421856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37afc8-fffd-4b50-a471-a2799b7b0824 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.425788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.426519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.426984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.427256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.440438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.441969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1327d6-da02-4692-abfe-270f61cbf4b3 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:48.454247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 979/1945] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:48.917153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:48.918633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:48.948274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99947c9-d00f-4783-8da8-92bbe4ac2330 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:48.953875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:48.955555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:48.956115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:48.956366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] PATH_INFO: `/v3/projects/fc704bcba15148bab9aeb3e422d5800e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:48.957986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc704bcba15148bab9aeb3e422d5800e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:48.963903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.141432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc704bcba15148bab9aeb3e422d5800e', 'request_id': 'req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.224019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc704bcba15148bab9aeb3e422d5800e', 'request_id': 'req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.231143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc704bcba15148bab9aeb3e422d5800e', 'request_id': 'req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.252056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] The token cache is being invalidate because project fc704bcba15148bab9aeb3e422d5800e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:49.252391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.253532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2d13f1a9-14b8-49d6-a76d-51bf2803ff14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:49.255847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 967/1946] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:48 2022] DELETE /identity/v3/projects/fc704bcba15148bab9aeb3e422d5800e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:49.260910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.261452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.261452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.287464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.289065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.372142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 980/1947] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:49.382082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.382564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.382641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.439642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:49.445870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:49.464159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 968/1948] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:49.477011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.489336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa6033-e1d4-461c-8b1e-64f4fcf26253 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.489336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.489336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.489336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.490084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.490084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:49.490084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.491391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.506835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 981/1949] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:49.511196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.527356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0b339-1e6d-4d42-8a5b-7a5c22b59935 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.533329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.533958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.534237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.534514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.535199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.537207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.697770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 969/1950] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:49.701589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.715030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7887b18-33f8-4466-8a40-66323956db6d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.810804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.810804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.810804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.810804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.811206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.812578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:49.972225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 982/1951] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:49 2022] POST /identity/v3/users => generated 283 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:49.974383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:49.986614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88930-622c-424a-895c-8b04f955b2d1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:49.991566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:49.992180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:49.992463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:49.992782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:49.993437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:49.995125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.006570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 970/1952] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:50.010775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.024712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef6f1-f7c0-4273-ba73-5b09150207d1 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.036239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.037190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.037472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.037749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.038434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.040487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.041661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.043760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.052672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 983/1953] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:50.056579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.060635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec393a-22ef-4ff7-9822-b02ec980c6d5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.066806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.067377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.067653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.067925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] PATH_INFO: `/v3/users/2aeb52d51d3f44319bb4e114858222f0` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.068353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147bdcd-63e1-40a7-8167-a27027305eae admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.069203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeb52d51d3f44319bb4e114858222f0)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.073521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.073774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.073774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.073989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.074260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] PATH_INFO: `/v3/projects/e68bfeb0ccd84947b9ccc0c41a89d58b/users/9404ac21e3204b149482aefa5b992088/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.077429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:50.089723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68bfeb0ccd84947b9ccc0c41a89d58b, user_id=9404ac21e3204b149482aefa5b992088, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.092574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320c790-a81e-47bf-ae1a-39ffb68c1380 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.250009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 984/1954] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:50 2022] PUT /identity/v3/projects/e68bfeb0ccd84947b9ccc0c41a89d58b/users/9404ac21e3204b149482aefa5b992088/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:50.260020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.260394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.260692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.289946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] MFA Rules not processed for user `9404ac21e3204b149482aefa5b992088`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:50.291255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.371313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 985/1955] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:50.376412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.378056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.381543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeb52d51d3f44319bb4e114858222f0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.392960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeb52d51d3f44319bb4e114858222f0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.409617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeb52d51d3f44319bb4e114858222f0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.410604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeb52d51d3f44319bb4e114858222f0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.411685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeb52d51d3f44319bb4e114858222f0'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.444320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837c06-32f7-4865-b4f5-ae29c7309e6a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.448309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.448960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.449242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.449521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.451281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.463225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.464579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.475049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 986/1956] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:50.475833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.477134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.518712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 971/1957] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:50 2022] DELETE /identity/v3/users/2aeb52d51d3f44319bb4e114858222f0 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:50.576286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:50.579098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:50.580874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926d87f-4a74-4021-be70-e0e07804b2e6 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.606149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:50.625171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:50.673950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a991-2e1f-482c-87e4-1b7e7cc20bc2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:50.679620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.680166 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.680607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.680896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] PATH_INFO: `/v3/users/c5f064c1e10e4fa58dbba823949f27fc` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.682063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f064c1e10e4fa58dbba823949f27fc)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.684035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.691218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:50.692908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:50.693188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:50.693460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] PATH_INFO: `/v3/users/2a375ff33f2c4e0092ab8d00aed7951d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:50.694656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a375ff33f2c4e0092ab8d00aed7951d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:50.696757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:50.885511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f064c1e10e4fa58dbba823949f27fc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.893279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f064c1e10e4fa58dbba823949f27fc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.906839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f064c1e10e4fa58dbba823949f27fc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.907633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f064c1e10e4fa58dbba823949f27fc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:50.908495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f064c1e10e4fa58dbba823949f27fc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.009601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a375ff33f2c4e0092ab8d00aed7951d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.017070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a375ff33f2c4e0092ab8d00aed7951d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.029215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a375ff33f2c4e0092ab8d00aed7951d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.029935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a375ff33f2c4e0092ab8d00aed7951d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.031551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-bf8bcebc-411d-4d46-85ab-56cbf96e4e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a375ff33f2c4e0092ab8d00aed7951d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.042065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 972/1958] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:50 2022] DELETE /identity/v3/users/c5f064c1e10e4fa58dbba823949f27fc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:51.043022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.044557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.109935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe2c5-396b-4760-9fd0-51ec442ef0d0 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.114927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.115611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.115890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.116164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] PATH_INFO: `/v3/projects/5515114a058944a49182063213de54d3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.117350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5515114a058944a49182063213de54d3)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.119258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.153913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 987/1959] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:50 2022] DELETE /identity/v3/users/2a375ff33f2c4e0092ab8d00aed7951d => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:51.155244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9071-6130-458f-8970-1bfa431dc8fc None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.155644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9071-6130-458f-8970-1bfa431dc8fc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.156149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9071-6130-458f-8970-1bfa431dc8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.175834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-de8e9071-6130-458f-8970-1bfa431dc8fc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:51.177172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-de8e9071-6130-458f-8970-1bfa431dc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.243807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 988/1960] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:51.243807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.243807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.243807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.255043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:51.263803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:51.265480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5515114a058944a49182063213de54d3', 'request_id': 'req-841fe724-9894-4b40-a430-1002bc1f99ea'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.265608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.308507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:51.328839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 989/1961] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:51.329671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.330965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.388933 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e396727-fe59-45cc-9b8c-84718534a1d0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.391103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5515114a058944a49182063213de54d3', 'request_id': 'req-841fe724-9894-4b40-a430-1002bc1f99ea'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.394569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.395251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.395836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.395836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.400465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5515114a058944a49182063213de54d3', 'request_id': 'req-841fe724-9894-4b40-a430-1002bc1f99ea'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.406091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.407077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.413644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] The token cache is being invalidate because project 5515114a058944a49182063213de54d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:51.414054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.416452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-841fe724-9894-4b40-a430-1002bc1f99ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:51.417770 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 990/1962] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:51.422744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.424134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.427219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 973/1963] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:51 2022] DELETE /identity/v3/projects/5515114a058944a49182063213de54d3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:51.429116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.429116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.429116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.435422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625225c2-bde0-43f3-94a8-035639058214 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.439936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.440548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.440863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.441136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.443148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.457999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.459758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17932c5-f5d4-4867-954c-266340a47734 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.461187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:51.463060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.469821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 991/1964] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:51.472136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 974/1965] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:51.479863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.485674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.485674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.506068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:51.507310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.517539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 992/1966] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:51.521779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.534674 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a57f-2817-4abe-a82b-ba70420e2ddb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.541873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.542710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.542925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.548971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.548971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:51.548971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.548971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.559347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 975/1967] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:51.564518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.574210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bb5ea-9c3a-4623-ba0f-e7933601e76b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.579212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.579920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.580270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.580638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.584723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.584723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.696925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 993/1968] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:51.697149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.705196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ed52-1356-4aa8-be75-2d5611fb57b0 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.762796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.763416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.763694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.763969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.764692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.767277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.841619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 976/1969] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:51 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:51.846145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.861980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce8429-5468-4bf1-8eb5-b2b85d4db26b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.872463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.873121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.873412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.873671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.874402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.876066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.888373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 994/1970] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:51.892613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.904651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0ebf2-118c-4476-8af3-0e99833e5d0b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.910520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.911131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.911417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.911717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.912353 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.913996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.929172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 977/1971] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:51.929911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.940417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f02c2-0397-4d9b-8f76-fc827375f4fc admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.945732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.946543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.946890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.947166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] PATH_INFO: `/v3/projects/794f655df3dd45aca9b501f1d5409de4/users/4466311c459c48a5b269d82dcfbfddf6/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.963581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:51.965712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:51.970782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794f655df3dd45aca9b501f1d5409de4, user_id=4466311c459c48a5b269d82dcfbfddf6, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.974097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:51.982013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec622a-9325-488a-b119-958ddd60c4f8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:51.988908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:51.989775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:51.990213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:51.990464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] PATH_INFO: `/v3/projects/69544a83ae2b4109b72f7621d8432d02` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:51.991607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69544a83ae2b4109b72f7621d8432d02)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:51.994608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.053633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 995/1972] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:51 2022] PUT /identity/v3/projects/794f655df3dd45aca9b501f1d5409de4/users/4466311c459c48a5b269d82dcfbfddf6/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:52.063991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.066042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.135753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f963-b00a-401f-b51f-8ed86d8d5c13 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.145266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69544a83ae2b4109b72f7621d8432d02', 'request_id': 'req-47776798-802a-4ef2-8043-22aba5042852'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.148419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:52.192196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.192818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.193121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.193397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] PATH_INFO: `/v3/projects/b5802203266940ac9b87f23d6b233904` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.194526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5802203266940ac9b87f23d6b233904)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.196485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.222612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69544a83ae2b4109b72f7621d8432d02', 'request_id': 'req-47776798-802a-4ef2-8043-22aba5042852'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.228901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69544a83ae2b4109b72f7621d8432d02', 'request_id': 'req-47776798-802a-4ef2-8043-22aba5042852'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.240374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] The token cache is being invalidate because project 69544a83ae2b4109b72f7621d8432d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:52.240687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.241366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47776798-802a-4ef2-8043-22aba5042852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.242879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 978/1973] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:51 2022] DELETE /identity/v3/projects/69544a83ae2b4109b72f7621d8432d02 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:52.244065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.244347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.244628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.257800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=84,keystone:DELETE=11,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:52.271806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] MFA Rules not processed for user `4466311c459c48a5b269d82dcfbfddf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:52.273053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.348954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5802203266940ac9b87f23d6b233904', 'request_id': 'req-4684a7df-59bc-45ef-a325-499d7cd15ae1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.353218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 979/1974] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:52.356160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.357687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.371464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8aaf5b-fe0e-4063-902b-d8bb3593cc63 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.430733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.431490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.432062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.432339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] PATH_INFO: `/v3/users/c08427fdc1974bc6b87a030cd9ee4423` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.434028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08427fdc1974bc6b87a030cd9ee4423)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.436836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.459290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5802203266940ac9b87f23d6b233904', 'request_id': 'req-4684a7df-59bc-45ef-a325-499d7cd15ae1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.465881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5802203266940ac9b87f23d6b233904', 'request_id': 'req-4684a7df-59bc-45ef-a325-499d7cd15ae1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.478565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] The token cache is being invalidate because project b5802203266940ac9b87f23d6b233904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:52.478861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.479561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.481133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 996/1975] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:52 2022] DELETE /identity/v3/projects/b5802203266940ac9b87f23d6b233904 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:52.481868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.483163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.495916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a7df-59bc-45ef-a325-499d7cd15ae1 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.500620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.501258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.501535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.501863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.502473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.504831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.611455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 997/1976] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:52 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:52.611455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.613456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.638405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08427fdc1974bc6b87a030cd9ee4423'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.656843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08427fdc1974bc6b87a030cd9ee4423'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.662503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08427fdc1974bc6b87a030cd9ee4423'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.663717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08427fdc1974bc6b87a030cd9ee4423'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.664610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08427fdc1974bc6b87a030cd9ee4423'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:52.681572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208632e-7efd-461e-aa29-61a58afb5d91 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.687443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.687806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.688090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.688348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] PATH_INFO: `/v3/users/5851dde37ba94ecdaef494b793970906` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.689539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5851dde37ba94ecdaef494b793970906)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.691446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:52.739123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 980/1977] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:52 2022] DELETE /identity/v3/users/c08427fdc1974bc6b87a030cd9ee4423 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:52.740284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:52.741946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:52.832521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b17ef-c1fb-4f71-b3a4-116cc4172d68 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:52.838831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:52.839596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:52.839961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:52.840364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:52.841163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:52.843206 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.014644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 981/1978] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:52 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:53.018513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.026608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5851dde37ba94ecdaef494b793970906'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.030172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07a8cb-54ed-4917-a63b-71a836972f77 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.034168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5851dde37ba94ecdaef494b793970906'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.036299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.037062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.037414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.037780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.038537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.040371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.047837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5851dde37ba94ecdaef494b793970906'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.048639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5851dde37ba94ecdaef494b793970906'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.049422 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5851dde37ba94ecdaef494b793970906'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.054228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 982/1979] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:53.060927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.064974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:53.081170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01150c4f-a59d-4218-a198-3ec9f00bde15 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.085905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.086571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.086850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.087124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] PATH_INFO: `/v3/projects/6c3de6dd8bb3453dbf21caa024084337/users/f7e81ba49bf447dc91a55e6649c2aea4/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.102301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c3de6dd8bb3453dbf21caa024084337, user_id=f7e81ba49bf447dc91a55e6649c2aea4, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.105045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.179483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 998/1980] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:52 2022] DELETE /identity/v3/users/5851dde37ba94ecdaef494b793970906 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:53.213573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:53.219650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 983/1981] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:53 2022] PUT /identity/v3/projects/6c3de6dd8bb3453dbf21caa024084337/users/f7e81ba49bf447dc91a55e6649c2aea4/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:53.226398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.281041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.282131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:53.290642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a644d11-f9fd-49ba-a503-241614cdc181 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.368687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.369423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.369661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.369935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.370581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.372235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.384782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 999/1982] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:53.388535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.398916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d0ae0-cc12-43de-872f-db4276497031 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.404649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.405312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.405835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.406294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] PATH_INFO: `/v3/projects/6c3de6dd8bb3453dbf21caa024084337/users/f7e81ba49bf447dc91a55e6649c2aea4/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.407104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e9f90-1863-4695-9957-446b506f7398 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.408288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c3de6dd8bb3453dbf21caa024084337, user_id=f7e81ba49bf447dc91a55e6649c2aea4, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.411003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cce0b-ca5a-41ef-8755-333e837ea63f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.411902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.412603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.413307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.413649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] PATH_INFO: `/v3/projects/f508881a9f1c4f8d94e5edf403c23f6e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.414851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f508881a9f1c4f8d94e5edf403c23f6e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.416917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.423580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1000/1983] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:53 2022] PUT /identity/v3/projects/6c3de6dd8bb3453dbf21caa024084337/users/f7e81ba49bf447dc91a55e6649c2aea4/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:53.435204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.435562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.435911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.485552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] MFA Rules not processed for user `f7e81ba49bf447dc91a55e6649c2aea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:53.486903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:53.567929 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1001/1984] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:53.575640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.578158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:53.578512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f508881a9f1c4f8d94e5edf403c23f6e', 'request_id': 'req-fe982a12-c876-4b94-b05b-955192c531f5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.646284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d0c1f-40f1-4dd1-88e7-638dbd09fe84 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.694521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.695140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.695420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.695730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.704006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f508881a9f1c4f8d94e5edf403c23f6e', 'request_id': 'req-fe982a12-c876-4b94-b05b-955192c531f5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.713187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f508881a9f1c4f8d94e5edf403c23f6e', 'request_id': 'req-fe982a12-c876-4b94-b05b-955192c531f5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.737408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] The token cache is being invalidate because project f508881a9f1c4f8d94e5edf403c23f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:53.737712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.738793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:53.743562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 984/1985] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:53 2022] DELETE /identity/v3/projects/f508881a9f1c4f8d94e5edf403c23f6e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:53.743562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.743562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:53.765528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.766999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.776651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1002/1986] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:53.779392 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:53.814745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe982a12-c876-4b94-b05b-955192c531f5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.820907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.821577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.821855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.822161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] PATH_INFO: `/v3/projects/4e554cc8aa064e7c89930ecef66846da` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.823665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e554cc8aa064e7c89930ecef66846da)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.825610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.847438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d1a7-3591-4518-9d94-741d7f69dd77 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:53.852757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:53.853412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:53.853680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:53.853960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] PATH_INFO: `/v3/users/c608d51158854c4db1e718c4161f60fc` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:53.855103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c608d51158854c4db1e718c4161f60fc)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:53.857155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:53.956597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e554cc8aa064e7c89930ecef66846da', 'request_id': 'req-8f0b9b11-9a6f-447e-85b8-b67894b98383'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.036130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e554cc8aa064e7c89930ecef66846da', 'request_id': 'req-8f0b9b11-9a6f-447e-85b8-b67894b98383'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.044844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e554cc8aa064e7c89930ecef66846da', 'request_id': 'req-8f0b9b11-9a6f-447e-85b8-b67894b98383'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.057827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] The token cache is being invalidate because project 4e554cc8aa064e7c89930ecef66846da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:54.058245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.059749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8f0b9b11-9a6f-447e-85b8-b67894b98383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.061827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 985/1987] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:53 2022] DELETE /identity/v3/projects/4e554cc8aa064e7c89930ecef66846da => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:54.068867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d8c7-5525-4c63-8a4b-42db5d080694 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.069305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d8c7-5525-4c63-8a4b-42db5d080694 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.075104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d8c7-5525-4c63-8a4b-42db5d080694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.094297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2fc7d8c7-5525-4c63-8a4b-42db5d080694 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:54.094297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2fc7d8c7-5525-4c63-8a4b-42db5d080694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.145726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c608d51158854c4db1e718c4161f60fc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.152386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 986/1988] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:54.153028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c608d51158854c4db1e718c4161f60fc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.160816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.161094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.161364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.166100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c608d51158854c4db1e718c4161f60fc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.166799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c608d51158854c4db1e718c4161f60fc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.167419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c608d51158854c4db1e718c4161f60fc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.187120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:54.188593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.196814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 987/1989] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:54.203957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.212917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bbb77-a26b-4620-baec-ba0c39818c95 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.217373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.218497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.218775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.219035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.219670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:54.219971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.221621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.237398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 988/1990] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:26:54.238456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.249322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2355-24de-4aef-8bc6-a02f1638f7ec admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.257465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.258139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.258481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.259005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.259733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.261204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1003/1991] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:53 2022] DELETE /identity/v3/users/c608d51158854c4db1e718c4161f60fc => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:54.261963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.406854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 989/1992] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:54.408773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.501468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239c68e-c17a-48a3-9bb7-20aba0431a4e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.557947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.558592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.558873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.559143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.559766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.561794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.596165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:54.697140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1004/1993] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/users => generated 290 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:54.701469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.712022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c96a9-e6bc-4e80-ae4e-04ea905321aa admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.716764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.717363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.717671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.718627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.719281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.720939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.737972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 990/1994] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:54.739408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.739767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.742119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:54.751111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f7cd5-45d5-41fc-a6dd-7cbfa94b67d9 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.756159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.756884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.757257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.757739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.758589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.760466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.773429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b51e7c-abee-48cc-85d9-c3391c062469 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.773760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 991/1995] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:54.780187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.781401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.781786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.782154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] PATH_INFO: `/v3/projects/8e89674d52c54167902f77d76fd2c28a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.783421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e89674d52c54167902f77d76fd2c28a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.784479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:54.785780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.795084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47533f64-f908-4e9b-afff-125dc86e8ee2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:54.800072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:54.800762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.801047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.801325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] PATH_INFO: `/v3/projects/7b3c46fd7fc747b6ae24b40dd2cecaf4/users/78e449f8df774e248fc323ad32bb599b/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.817691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3c46fd7fc747b6ae24b40dd2cecaf4, user_id=78e449f8df774e248fc323ad32bb599b, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:54.820355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015b37e-f66d-4e6e-a553-1971e4a8e7a3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:54.918362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 992/1996] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:54 2022] PUT /identity/v3/projects/7b3c46fd7fc747b6ae24b40dd2cecaf4/users/78e449f8df774e248fc323ad32bb599b/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:54.930335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e89674d52c54167902f77d76fd2c28a', 'request_id': 'req-73c0413b-6a53-40d0-839e-2515f2fb3e35'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:54.932328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:54.932993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:54.932993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:54.961811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] MFA Rules not processed for user `78e449f8df774e248fc323ad32bb599b`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:54.963014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.001895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e89674d52c54167902f77d76fd2c28a', 'request_id': 'req-73c0413b-6a53-40d0-839e-2515f2fb3e35'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.009602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e89674d52c54167902f77d76fd2c28a', 'request_id': 'req-73c0413b-6a53-40d0-839e-2515f2fb3e35'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.031588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 993/1997] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:55.033652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.034825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] The token cache is being invalidate because project 8e89674d52c54167902f77d76fd2c28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:55.035472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.035786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.036550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.038395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1005/1998] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:54 2022] DELETE /identity/v3/projects/8e89674d52c54167902f77d76fd2c28a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:55.039099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.040649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.101560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fe45e-14ce-4b2f-8bb9-7c79b55c2359 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.108210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.113175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0413b-6a53-40d0-839e-2515f2fb3e35 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.113808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.113808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.113808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] PATH_INFO: `/v3/users/4466311c459c48a5b269d82dcfbfddf6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.113808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4466311c459c48a5b269d82dcfbfddf6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.113808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.114740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.115435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.115824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.116195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.116962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.118983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43105df7-d748-474b-a4a7-34f214ce428d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.248628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1006/1999] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:55.250192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.250468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.250742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.271952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:55.273445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.336119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1007/2000] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:55.337238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.346491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4466311c459c48a5b269d82dcfbfddf6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.348421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d253a1d-1246-42ab-b95b-0655e431922b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.353592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4466311c459c48a5b269d82dcfbfddf6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.353723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.354348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.354636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.354944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.355540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.357502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a10dd5-8a7f-4662-a2c1-6b3fb7e66a88 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.369575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4466311c459c48a5b269d82dcfbfddf6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.369575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4466311c459c48a5b269d82dcfbfddf6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.371122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-64058305-5eac-4a62-86d6-fa2545baaa40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4466311c459c48a5b269d82dcfbfddf6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:55.453838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:55.487386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 994/2001] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:55 2022] DELETE /identity/v3/users/4466311c459c48a5b269d82dcfbfddf6 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:55.490805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1008/2002] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:55.491042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.491042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.491042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.492761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.493070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.493344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.516830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:55.517170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:55.518661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.518778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.583086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 995/2003] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:55.583715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.585046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.586605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1009/2004] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:55.588188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.599384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cbd7-cbd1-4cb7-bf2d-4c58df9b2bca None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.607932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.608847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f5377-4812-4142-bf0b-f35b0de0a322 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.608971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.609227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.609511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.610174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.611836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.613750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.614357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.614631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.614899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.615543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:55.615844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.617554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.622530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 996/2005] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:55.623165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.625362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:55.634076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1010/2006] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:55.634076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.642114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d149d-fde8-47b4-88c6-366fe7704ec8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.646741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.647387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.647664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.647937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] PATH_INFO: `/v3/projects/7a7a6810280b49efb46df6e9f70f66d8/users/aac93573cce545668ebdad6d95d02d0b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.662472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7a6810280b49efb46df6e9f70f66d8, user_id=aac93573cce545668ebdad6d95d02d0b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.665218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.682829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34848ff0-cfd8-4995-986f-1080b5dedb29 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.687018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.687970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.688295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.688572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.697920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.699419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.709338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 997/2007] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:55.710101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.718934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cca46-e44b-49bd-b0aa-5863059a6759 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.723675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.724199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.724478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.724786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.725412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.727284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.761947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1011/2008] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:55 2022] PUT /identity/v3/projects/7a7a6810280b49efb46df6e9f70f66d8/users/aac93573cce545668ebdad6d95d02d0b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:55.766502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.830129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d500e4-ee67-4299-9144-6326992e2903 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.834925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.835536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.836088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.836378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.837069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.838688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.848582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 998/2009] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:55.849733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1012/2010] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:55.853009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.855217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:55.865579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd36c4b-0b85-47e8-b2e8-3a9816e2bed9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.927417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7317b78-f37a-4611-915f-ee52cbac4081 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:55.932241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.933090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.933436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.935107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.935828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.946376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.953695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:55.954398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.954668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.954940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] PATH_INFO: `/v3/projects/7a7a6810280b49efb46df6e9f70f66d8/users/aac93573cce545668ebdad6d95d02d0b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:55.956978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7a6810280b49efb46df6e9f70f66d8, user_id=aac93573cce545668ebdad6d95d02d0b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:55.959642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7594b-f305-4975-92ea-2ff657732fc4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:55.971859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1013/2011] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:55 2022] PUT /identity/v3/projects/7a7a6810280b49efb46df6e9f70f66d8/users/aac93573cce545668ebdad6d95d02d0b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:55.974395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:55.981832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:55.982110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:55.982702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.018539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] MFA Rules not processed for user `aac93573cce545668ebdad6d95d02d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:56.019916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.070829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 999/2012] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:56.074802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.084472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11340d64-7b64-447c-ad40-066642cd4439 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.104287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1014/2013] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:26:56.142559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.143087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.143365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.143634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.144270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.145909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.155326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1000/2014] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:26:56.159548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.169644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83d49d-3ae0-445d-95d1-2544dcf8a2a4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.174869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.175464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.175740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.176009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.176692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.178288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.190407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1015/2015] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:56.193349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.202365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.203650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998811-134f-45ba-925d-ac7e325e33f1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.205209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.208522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.209413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.209509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.209793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] PATH_INFO: `/v3/projects/f4135d208494495caa74cf2c6a939050/users/31a702cac12d447ab96ea03d85ce969f/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.231629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446209b8-1810-4603-9f39-5dea65d2912d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.235043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4135d208494495caa74cf2c6a939050, user_id=31a702cac12d447ab96ea03d85ce969f, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.237764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.241200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.242123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.242406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.242683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] PATH_INFO: `/v3/projects/794f655df3dd45aca9b501f1d5409de4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.243824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794f655df3dd45aca9b501f1d5409de4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.245927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.356091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1001/2016] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:56 2022] PUT /identity/v3/projects/f4135d208494495caa74cf2c6a939050/users/31a702cac12d447ab96ea03d85ce969f/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:26:56.357026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.359728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.398872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794f655df3dd45aca9b501f1d5409de4', 'request_id': 'req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.419578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5836d7-7fa9-4988-bcd6-ad1e1febea91 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.469972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.470602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.470885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.471163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.475439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794f655df3dd45aca9b501f1d5409de4', 'request_id': 'req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.481762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794f655df3dd45aca9b501f1d5409de4', 'request_id': 'req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.494720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] The token cache is being invalidate because project 794f655df3dd45aca9b501f1d5409de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:56.495034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.495844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-20b3975f-38b5-4cd5-82e7-ce6c291cd7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.497389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1016/2017] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:56 2022] DELETE /identity/v3/projects/794f655df3dd45aca9b501f1d5409de4 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:56.498683 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.499085 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.499447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.529674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] MFA Rules not processed for user `31a702cac12d447ab96ea03d85ce969f`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:56.531591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.535807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.537369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.547877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1002/2018] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:56.548784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.602160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1017/2019] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:56.611990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419ec3c-a140-4353-aeb4-5f7e35e764e0 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.614363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.615752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.616711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.617350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.617631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.617908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] PATH_INFO: `/v3/users/f7e81ba49bf447dc91a55e6649c2aea4` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.619050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e81ba49bf447dc91a55e6649c2aea4)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.621027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.628319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708040f-08a9-4751-84d6-ccdf93b7cb9b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.632527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.633181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.633464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.633848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.643233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.644758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.659931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1018/2020] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:56.674675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:56.826381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:56.827840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.839632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ac3d6-e4e0-4c83-b724-b5b7d2ff0136 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:56.843767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:56.844562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:56.844707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:56.845027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:56.847045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:56.849565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e81ba49bf447dc91a55e6649c2aea4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.856499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e81ba49bf447dc91a55e6649c2aea4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.869639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e81ba49bf447dc91a55e6649c2aea4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.870367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e81ba49bf447dc91a55e6649c2aea4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.871373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6cdd62a2-3f65-40c4-906a-a98fd48e745c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e81ba49bf447dc91a55e6649c2aea4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:56.910864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:56.912331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:56.922009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1019/2021] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:26:57.019151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1003/2022] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:26:56 2022] DELETE /identity/v3/users/f7e81ba49bf447dc91a55e6649c2aea4 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:57.121714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:57.280690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:57.431630 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.433376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.448307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29772aeb-83d3-45c5-9126-0d35b5f9af65 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.512202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.512898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.513133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.513404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] PATH_INFO: `/v3/projects/6c3de6dd8bb3453dbf21caa024084337` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.515041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c3de6dd8bb3453dbf21caa024084337)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.517050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.648293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c3de6dd8bb3453dbf21caa024084337', 'request_id': 'req-5ed6def5-1ec2-43eb-8596-07d49ddb2221'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:57.732783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c3de6dd8bb3453dbf21caa024084337', 'request_id': 'req-5ed6def5-1ec2-43eb-8596-07d49ddb2221'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:57.741003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c3de6dd8bb3453dbf21caa024084337', 'request_id': 'req-5ed6def5-1ec2-43eb-8596-07d49ddb2221'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:57.753845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] The token cache is being invalidate because project 6c3de6dd8bb3453dbf21caa024084337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:26:57.759114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:57.759114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5ed6def5-1ec2-43eb-8596-07d49ddb2221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:26:57.759114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1020/2023] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:26:57 2022] DELETE /identity/v3/projects/6c3de6dd8bb3453dbf21caa024084337 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:57.762186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.762656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.762776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.784931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:57.786278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.845906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1004/2024] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:57.853678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.853971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.854252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.877002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:57.878299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:57.886753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1021/2025] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:57.891296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.901063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c0083-68bd-41bc-8692-2ba4bac66466 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.906440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.907049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.907329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.907602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.908313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:26:57.908557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.910187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:57.922248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1005/2026] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:26:57.928141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:57.935839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c928008-149e-47da-aaad-9b47c91fa78d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:57.940944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:57.941536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:57.941814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:57.942087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:57.942699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:57.944717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.098118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1022/2027] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:57 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:58.099267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.110475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3f11e-471e-412c-b4ce-5c3908c6504a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:58.187071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:58.188224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.195867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.195867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.195867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:58.195867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.292077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1006/2028] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:58 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:26:58.293431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.307129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be3ce8-31d7-48dc-a7fc-6272f29ee66e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:58.316525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:58.317191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.317474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.317749 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.318403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:58.320622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.332961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1023/2029] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:58.337220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.352060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e32d3-916c-43f5-b049-1461ed2eb1cf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:58.354074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:58.357505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.357831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.358271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.359332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:58.361296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7be62-e71c-47c8-8ac4-23c0a1e92378 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.376799 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1007/2030] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:58.378221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.391829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f0e07-3e07-414e-b0ec-721125b48aa3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:58.400882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:58.400882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.400882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.400882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] PATH_INFO: `/v3/projects/804850c9a1d043fe9b08017a77362eea/users/0e199c137c2c4d889ef8a0f33557a5a6/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.417021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804850c9a1d043fe9b08017a77362eea, user_id=0e199c137c2c4d889ef8a0f33557a5a6, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:58.419924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.592508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1024/2031] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:58 2022] PUT /identity/v3/projects/804850c9a1d043fe9b08017a77362eea/users/0e199c137c2c4d889ef8a0f33557a5a6/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:58.602939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.603227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.603502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.633095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] MFA Rules not processed for user `0e199c137c2c4d889ef8a0f33557a5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:58.634479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:58.735676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1008/2032] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:58.738292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.816399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b8cf-73a9-46a9-9031-3a885d4db5a4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:58.821109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:58.821761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:58.822038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:58.822311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:58.822927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:58.824848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:58.898453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:58.978997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1025/2033] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:26:58 2022] POST /identity/v3/projects => generated 393 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:58.982986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:58.995066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1722c-35c2-4b5f-8e8f-7c54ec3bc91d None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.049430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.050042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.050321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.050594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.051221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.053195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.222190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1009/2034] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:26:58 2022] POST /identity/v3/users => generated 303 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:26:59.223340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.235340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32143a56-e7a7-46f1-af1b-5b156550e492 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.239843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.240445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.240758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.241032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.241672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.243245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.254817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1026/2035] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:59.259146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.269332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a02e62-ba6e-4b18-ae79-c9169291adcd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.274281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.275184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.275754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.278867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] PATH_INFO: `/v3/projects/5df07d21f119485aa85331929fa98941/users/6d52b235278f4c8fbce1fda468e4f263/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.279253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:26:59.302915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df07d21f119485aa85331929fa98941, user_id=6d52b235278f4c8fbce1fda468e4f263, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.306012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.397951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1010/2036] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:59 2022] PUT /identity/v3/projects/5df07d21f119485aa85331929fa98941/users/6d52b235278f4c8fbce1fda468e4f263/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:59.403394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.414187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eb73c-5b0d-43e0-b951-c13d22890941 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.471361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.471990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.472271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.472547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.473236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.474874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070308d-ce4b-4503-a408-3ce26728f56f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.485684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1027/2037] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:26:59.489780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.515937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f9eda-44f5-474c-bcb5-016814c91c0f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.525190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917b37-96c1-4161-8880-164102845adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.526414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917b37-96c1-4161-8880-164102845adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.526764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917b37-96c1-4161-8880-164102845adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.527918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81917b37-96c1-4161-8880-164102845adf admin admin] PATH_INFO: `/v3/projects/5df07d21f119485aa85331929fa98941/users/6d52b235278f4c8fbce1fda468e4f263/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.534881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81917b37-96c1-4161-8880-164102845adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df07d21f119485aa85331929fa98941, user_id=6d52b235278f4c8fbce1fda468e4f263, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.537713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81917b37-96c1-4161-8880-164102845adf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.550942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1011/2038] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:26:59 2022] PUT /identity/v3/projects/5df07d21f119485aa85331929fa98941/users/6d52b235278f4c8fbce1fda468e4f263/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:26:59.578655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.578975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.579273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.618175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] MFA Rules not processed for user `6d52b235278f4c8fbce1fda468e4f263`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:26:59.619685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:59.704376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1028/2039] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:26:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:26:59.715567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917b37-96c1-4161-8880-164102845adf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.717314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-81917b37-96c1-4161-8880-164102845adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:26:59.775797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81917b37-96c1-4161-8880-164102845adf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.780974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.781565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.781845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.782483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.792111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.793677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.803273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1012/2040] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:26:59.860270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:26:59.869752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda4211-82c9-4dc9-877d-4476719bc688 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:26:59.873513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:26:59.874720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:26:59.875000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:26:59.875266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:26:59.932820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:26:59.934252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ff860-36f3-4dac-9960-7f6f89802117 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:26:59.943305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1029/2041] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:26:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:01.155435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:01.205887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:01.595056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:01.744464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:01.961136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:01.989743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:02.087051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:02.195379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:12.291266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=73 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:12.321297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:14.137535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:14.202711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cbae0-6709-4377-8190-0a5b6342367c service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:14.207417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:14.208082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:14.208363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:14.208639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] PATH_INFO: `/v3/users/0e199c137c2c4d889ef8a0f33557a5a6` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:14.209792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e199c137c2c4d889ef8a0f33557a5a6)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:14.211722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:14.496595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e199c137c2c4d889ef8a0f33557a5a6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:14.504400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e199c137c2c4d889ef8a0f33557a5a6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:14.519244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e199c137c2c4d889ef8a0f33557a5a6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:14.519244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e199c137c2c4d889ef8a0f33557a5a6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:14.519622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e199c137c2c4d889ef8a0f33557a5a6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:14.613677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1013/2042] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:14 2022] DELETE /identity/v3/users/0e199c137c2c4d889ef8a0f33557a5a6 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:14.618510 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:14.619011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:14.619112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:14.646251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:14.647588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:14.708495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1030/2043] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:14.719266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:14.720685 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:14.786373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcea5d-478a-45dd-a8c4-6e498c8d126e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:14.786373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:14.786868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:14.787215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:14.787493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:14.796980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:14.798402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:14.807706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1014/2044] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:15.317640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:15.319041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:15.334823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982c4b0-50ab-47b8-9305-9de22d853e4c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:15.337182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:15.337817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:15.338265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:15.338547 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] PATH_INFO: `/v3/projects/804850c9a1d043fe9b08017a77362eea` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:15.339694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804850c9a1d043fe9b08017a77362eea)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:15.341702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:15.455222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804850c9a1d043fe9b08017a77362eea', 'request_id': 'req-3a6b8af1-3dad-469d-8377-5f79a58547a4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.610344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804850c9a1d043fe9b08017a77362eea', 'request_id': 'req-3a6b8af1-3dad-469d-8377-5f79a58547a4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.616521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804850c9a1d043fe9b08017a77362eea', 'request_id': 'req-3a6b8af1-3dad-469d-8377-5f79a58547a4'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.633892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] The token cache is being invalidate because project 804850c9a1d043fe9b08017a77362eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:15.633892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.633892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.633892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1031/2045] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:15 2022] DELETE /identity/v3/projects/804850c9a1d043fe9b08017a77362eea => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:15.634734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:15.644692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066d5e7-981f-4431-845e-a4a3c5b11ad8 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:15.664655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:15.703369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:15.704007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:15.704289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:15.704565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] PATH_INFO: `/v3/users/6d52b235278f4c8fbce1fda468e4f263` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:15.705736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d52b235278f4c8fbce1fda468e4f263)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:15.707688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:15.891409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d52b235278f4c8fbce1fda468e4f263'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.898418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d52b235278f4c8fbce1fda468e4f263'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.909951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d52b235278f4c8fbce1fda468e4f263'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.910670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d52b235278f4c8fbce1fda468e4f263'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.916956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-26da1a29-632c-4e7a-8bf1-cb6d8d6b21ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d52b235278f4c8fbce1fda468e4f263'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:15.995296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1015/2046] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:15 2022] DELETE /identity/v3/users/6d52b235278f4c8fbce1fda468e4f263 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:16.409485 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:16.411012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:16.431226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b8af1-3dad-469d-8377-5f79a58547a4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:16.485887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:16.486519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:16.486792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:16.487062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] PATH_INFO: `/v3/projects/5df07d21f119485aa85331929fa98941` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:16.488187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df07d21f119485aa85331929fa98941)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:16.490125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:16.620173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:16.636490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df07d21f119485aa85331929fa98941', 'request_id': 'req-c7f0565e-2fa7-4fab-9aff-35de5992611c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:16.732747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df07d21f119485aa85331929fa98941', 'request_id': 'req-c7f0565e-2fa7-4fab-9aff-35de5992611c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:16.739755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df07d21f119485aa85331929fa98941', 'request_id': 'req-c7f0565e-2fa7-4fab-9aff-35de5992611c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:16.751810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] The token cache is being invalidate because project 5df07d21f119485aa85331929fa98941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:16.752103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:16.752779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c7f0565e-2fa7-4fab-9aff-35de5992611c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:16.754338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1032/2047] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:16 2022] DELETE /identity/v3/projects/5df07d21f119485aa85331929fa98941 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:16.759112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:16.759589 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:16.759722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:16.778982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:16.780727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:16.843440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1016/2048] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:16.851554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:16.851853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:16.852142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:16.874755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:16.876441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:16.887732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1033/2049] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:16.892328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:16.902634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4678-bae0-4c40-a6be-350964e34e94 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:16.908221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:16.908926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:16.909204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:16.909533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:16.910406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:16.910912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:16.912496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:16.924953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1017/2050] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:16.929343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:16.939250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e253d-9126-408b-8620-afdfc16f517d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:16.944099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:16.945289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:16.945289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:16.945463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:16.946088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:16.948155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.117024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1034/2051] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:16 2022] POST /identity/v3/projects => generated 399 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:17.121222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.131118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaaf712-2448-4421-827d-813e16e6d01f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.168582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:17.192061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.192717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.192994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.193294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.193969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.195902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.291667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1018/2052] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:17 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:17.294254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.304811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b9ac-3df1-480f-880e-e1170e303c82 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.309551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.310180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.310459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.310734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.311383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.313841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.324275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1035/2053] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:17.329240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.339190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea74f75-75cb-423a-bed1-2337c32c55e4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.343858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.344456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.344782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.345058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.345732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.347349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080244d-9835-49f9-be56-03d9e3f10ce8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.357859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1019/2054] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:17.361344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.371276 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394f2f9-81ca-41e7-9df2-0a99dc4ddfa6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.376687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.377360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.377635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.377905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] PATH_INFO: `/v3/projects/d6e67c98bad5401f97ef77e168d8d358/users/da1bee34ddec4286bd6085e6de14d9cc/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.393081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e67c98bad5401f97ef77e168d8d358, user_id=da1bee34ddec4286bd6085e6de14d9cc, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.395733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.539721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1036/2055] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:17 2022] PUT /identity/v3/projects/d6e67c98bad5401f97ef77e168d8d358/users/da1bee34ddec4286bd6085e6de14d9cc/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:17.555887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.555887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.555887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.609888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] MFA Rules not processed for user `da1bee34ddec4286bd6085e6de14d9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:17.609888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:17.716677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1020/2056] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:17.721931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.807344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f213-1122-4394-a777-b865d68b9169 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:17.822686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:17.823625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:17.823822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:17.824135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:17.825003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:17.826716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:17.944875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1037/2057] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:17 2022] POST /identity/v3/projects => generated 401 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:17.949377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:17.961376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92f682-0431-4170-a1b7-004fbef91772 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.055030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.055724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.056074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.056488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.057182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.059204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.208218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1021/2058] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:17 2022] POST /identity/v3/users => generated 307 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:18.212049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:18.226542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c9d53-f8bc-490c-9067-bc3c2a9d3e95 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.237686 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.238411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.238694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.238963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.239613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.241355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.253610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1038/2059] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:18.257910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:18.269876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f309f-e703-4f98-bf71-d466e97d7cfd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.276059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.277358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.277699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.278040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] PATH_INFO: `/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f/users/868443c5e16d4a8ab31474884e7b645d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.295066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe6ff5e8cb14a799d944d82d5a76f8f, user_id=868443c5e16d4a8ab31474884e7b645d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.298507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.373846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1022/2060] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:18 2022] PUT /identity/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f/users/868443c5e16d4a8ab31474884e7b645d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:18.378234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:18.393481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745dca3-1583-4f6e-998f-5c53de83a9c6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.414221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:18.452905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.453512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.453789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.454059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.454696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.456274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427734aa-47c1-496f-b8bd-ea82544df760 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.465997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1039/2061] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:18.470365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:18.475746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:18.480989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa870-851c-4039-b96a-5f8bc3eb3351 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.487102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.487774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.488056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.488332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] PATH_INFO: `/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f/users/868443c5e16d4a8ab31474884e7b645d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.490357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe6ff5e8cb14a799d944d82d5a76f8f, user_id=868443c5e16d4a8ab31474884e7b645d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.493072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.504073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1023/2062] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:18 2022] PUT /identity/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f/users/868443c5e16d4a8ab31474884e7b645d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:18.514142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.514433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.514948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.543104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] MFA Rules not processed for user `868443c5e16d4a8ab31474884e7b645d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:18.544330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:18.617464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1040/2063] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:18 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:18.637986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:18.641065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:18.698515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a255a9-0074-4db1-9107-64f854fce8eb admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:18.702598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:18.703201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:18.703479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:18.703750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:18.712498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:18.713951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:18.724202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1024/2064] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:18 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:19.150947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:19.160782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeab1d-b83b-42c4-90f5-a78ccbdc48cc None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:19.164289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:19.164938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:19.165223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:19.165633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:19.228133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:19.231600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3599-a572-4e68-bb5e-966d71da4001 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:19.239760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1041/2065] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:19.549579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:19.551183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:19.628455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fac004-8a47-48cc-b4ce-e37c38db0198 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:19.632930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:19.633556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:19.633890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:19.634169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] PATH_INFO: `/v3/users/78e449f8df774e248fc323ad32bb599b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:19.635293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e449f8df774e248fc323ad32bb599b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:19.637245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:19.941955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e449f8df774e248fc323ad32bb599b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:19.950434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e449f8df774e248fc323ad32bb599b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:19.967325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e449f8df774e248fc323ad32bb599b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:19.969266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e449f8df774e248fc323ad32bb599b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:19.969949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e449f8df774e248fc323ad32bb599b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:20.114900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1025/2066] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:19 2022] DELETE /identity/v3/users/78e449f8df774e248fc323ad32bb599b => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:20.119577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.119952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.120296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.141640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:20.142878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.203146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:20.229848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1042/2067] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:20.241102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:20.245793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.259732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:20.333573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba7799-1fab-4687-a360-af62dfddbfdf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:20.338515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:20.339255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.339625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.339997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.352361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:20.354055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:20.373575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1026/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:20.895633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:20.898604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:20.912751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78182ff4-b7ab-464e-a503-7e7605333f6f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:20.919316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:20.919957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:20.920240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:20.920515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] PATH_INFO: `/v3/projects/7b3c46fd7fc747b6ae24b40dd2cecaf4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:20.926652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3c46fd7fc747b6ae24b40dd2cecaf4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:20.926652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:21.071691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3c46fd7fc747b6ae24b40dd2cecaf4', 'request_id': 'req-3bf99fae-ba91-43c9-89e6-480e7228f902'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.178270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3c46fd7fc747b6ae24b40dd2cecaf4', 'request_id': 'req-3bf99fae-ba91-43c9-89e6-480e7228f902'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.185166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3c46fd7fc747b6ae24b40dd2cecaf4', 'request_id': 'req-3bf99fae-ba91-43c9-89e6-480e7228f902'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.198198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] The token cache is being invalidate because project 7b3c46fd7fc747b6ae24b40dd2cecaf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:21.198486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.199138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.200934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1043/2069] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:20 2022] DELETE /identity/v3/projects/7b3c46fd7fc747b6ae24b40dd2cecaf4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:21.204968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:21.214636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a664a-9320-4ce7-94ca-d6481e6c7dc6 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:21.283037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:21.283671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:21.283948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:21.284218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] PATH_INFO: `/v3/users/aac93573cce545668ebdad6d95d02d0b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:21.286365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac93573cce545668ebdad6d95d02d0b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:21.288490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:21.563011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac93573cce545668ebdad6d95d02d0b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.570231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac93573cce545668ebdad6d95d02d0b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.582917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac93573cce545668ebdad6d95d02d0b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.583530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac93573cce545668ebdad6d95d02d0b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.584142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-670b07ca-f25f-4c18-8c66-db99d7c9e71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac93573cce545668ebdad6d95d02d0b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:21.722077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1027/2070] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:21 2022] DELETE /identity/v3/users/aac93573cce545668ebdad6d95d02d0b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:22.226529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:22.228658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:22.264242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf99fae-ba91-43c9-89e6-480e7228f902 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:22.340611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:22.341370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.341834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.342113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] PATH_INFO: `/v3/projects/7a7a6810280b49efb46df6e9f70f66d8` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.344035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7a6810280b49efb46df6e9f70f66d8)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:22.356809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:22.589395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7a6810280b49efb46df6e9f70f66d8', 'request_id': 'req-5321ded0-dc7b-4e27-b9d9-40951da66ba8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:22.772558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7a6810280b49efb46df6e9f70f66d8', 'request_id': 'req-5321ded0-dc7b-4e27-b9d9-40951da66ba8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:22.779561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7a6810280b49efb46df6e9f70f66d8', 'request_id': 'req-5321ded0-dc7b-4e27-b9d9-40951da66ba8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:22.788585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:22.793427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] The token cache is being invalidate because project 7a7a6810280b49efb46df6e9f70f66d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:22.793750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:22.794420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5321ded0-dc7b-4e27-b9d9-40951da66ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:22.796542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1044/2071] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:22 2022] DELETE /identity/v3/projects/7a7a6810280b49efb46df6e9f70f66d8 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:22.802593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.803156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.803268 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.824602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:22.825847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:22.856567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:22.890373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1028/2072] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:22.897984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56935db9-fbed-442b-9235-36342e95075e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.898307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56935db9-fbed-442b-9235-36342e95075e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.898813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56935db9-fbed-442b-9235-36342e95075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.921429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-56935db9-fbed-442b-9235-36342e95075e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:22.922650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-56935db9-fbed-442b-9235-36342e95075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:22.930992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1045/2073] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:22.935504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:22.947135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3439423-6a84-44d1-954d-6bf4da0310b2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:22.953423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:22.953423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.953423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.953752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.954362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:22.954682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:22.956276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:22.968803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1029/2074] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:22.973254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56935db9-fbed-442b-9235-36342e95075e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:22.982834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56935db9-fbed-442b-9235-36342e95075e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:22.988742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:22.989519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:22.989978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:22.990370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:22.991056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:22.993634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.150222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1046/2075] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:22 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:23.151683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:23.163887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f519-041a-4f73-9f76-1ee8fb40ad93 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.237087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.238421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.238810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.239096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.239888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.242034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.328158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1030/2076] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:23 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:23.332761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:23.344990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359af8d4-0eb4-48ea-825b-4842fc08abc5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.350875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.354240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.354240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.354240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.354240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.354799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.365437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1047/2077] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:23.370403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:23.381781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7c63-c647-44ed-abf1-9f7178a71bbd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.386451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.387048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.387324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.387595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.388237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.390615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.402876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1031/2078] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:23.407156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:23.419191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a64ac-542b-4523-8593-0de4baca9d1c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.425996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.426664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.426941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.427211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] PATH_INFO: `/v3/projects/f50300d61f164b9aa7c9866eb5d84bde/users/e971d6fa9eb54088bf7978821653cee1/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.451236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50300d61f164b9aa7c9866eb5d84bde, user_id=e971d6fa9eb54088bf7978821653cee1, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.455249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99eedb-fae6-467a-aad7-0ce66e7b29aa admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.486282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:23.486282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:23.488885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577797-9be0-4fbe-84d8-45fc189690c8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:23.494051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:23.494794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.495139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.495504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] PATH_INFO: `/v3/users/31a702cac12d447ab96ea03d85ce969f` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.496966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a702cac12d447ab96ea03d85ce969f)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:23.499930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:23.870523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1048/2079] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:23 2022] PUT /identity/v3/projects/f50300d61f164b9aa7c9866eb5d84bde/users/e971d6fa9eb54088bf7978821653cee1/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:23.882572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:23.883098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:23.883379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:23.917603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] MFA Rules not processed for user `e971d6fa9eb54088bf7978821653cee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:23.918883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:23.977664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a702cac12d447ab96ea03d85ce969f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:23.986273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a702cac12d447ab96ea03d85ce969f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:23.989223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1049/2080] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:23.993883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.000995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a702cac12d447ab96ea03d85ce969f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:24.001771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a702cac12d447ab96ea03d85ce969f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:24.002478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-47f9ba87-9b38-4d01-9ae3-fb1f65438120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a702cac12d447ab96ea03d85ce969f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:24.060347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effe636-8472-4039-a545-e0d88b3bb1b8 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.066384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.067000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.067315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.067594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.068319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.070380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.110295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1032/2081] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:23 2022] DELETE /identity/v3/users/31a702cac12d447ab96ea03d85ce969f => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:24.114956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.115248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.115583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.136597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:24.137990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:24.144201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:24.204555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1033/2082] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:24.211768 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1050/2083] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:23 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:24.216784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.219178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:24.223964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.226615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:24.243699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bc261-95dd-4b5a-bd99-d7a301cbb8da None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.248858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:24.301058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.301364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.301569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.301847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.302554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.304583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.305531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae5642-60aa-4be3-8982-0927fe88a398 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.310168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.311019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.311300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.312426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.326316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.329436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.340976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1051/2084] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:24.390654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1034/2085] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:24 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:24.395343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.406730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679b8dd-ab43-4bf0-bb3e-6105eaf0778b service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.412531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.413353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.413797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.414161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.414924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.420016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.434536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1052/2086] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:24.435837 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.450217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f3e3b-606f-412c-8e1a-3b8610bafb07 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.458040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.458830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.459092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.459423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] PATH_INFO: `/v3/projects/09ade170d87249f680892b2213e8b643/users/e1559eb1fe7649e58e4d0ad89e2ab486/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.476542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ade170d87249f680892b2213e8b643, user_id=e1559eb1fe7649e58e4d0ad89e2ab486, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.479549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.563611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1035/2087] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:24 2022] PUT /identity/v3/projects/09ade170d87249f680892b2213e8b643/users/e1559eb1fe7649e58e4d0ad89e2ab486/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:24.567767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.590012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dacf32-03f3-486c-8b49-7ed3c1e8eadf admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.730549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.731634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.732233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.732780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.734430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.736996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610eb1-ee86-4b34-ad55-0ce6692ad776 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.762164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1053/2088] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:24.762339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:24.779992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66cb09-7208-4ddd-84b3-a5734a7deabd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:24.785849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:24.786248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.786751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.787100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] PATH_INFO: `/v3/projects/09ade170d87249f680892b2213e8b643/users/e1559eb1fe7649e58e4d0ad89e2ab486/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.791173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ade170d87249f680892b2213e8b643, user_id=e1559eb1fe7649e58e4d0ad89e2ab486, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:24.795484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:24.817467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1036/2089] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:24 2022] PUT /identity/v3/projects/09ade170d87249f680892b2213e8b643/users/e1559eb1fe7649e58e4d0ad89e2ab486/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:24.822687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:24.822687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:24.822687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:24.867665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] MFA Rules not processed for user `e1559eb1fe7649e58e4d0ad89e2ab486`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:24.869878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:24.996292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1054/2090] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:25.032737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:25.035204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.116507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:25.118245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.125458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9150-0dfe-4b3e-b735-7cfdb7a55014 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:25.130789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:25.131535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.131828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.132104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] PATH_INFO: `/v3/projects/f4135d208494495caa74cf2c6a939050` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.133291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4135d208494495caa74cf2c6a939050)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:25.135306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:25.197032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735403d7-b9a8-425c-a676-d2f8b6e3c497 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:25.203699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:25.203699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.203699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.204126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.205867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.290344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:25.291107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4135d208494495caa74cf2c6a939050', 'request_id': 'req-f0383894-9129-4cb6-948e-1bca390870fa'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.292187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:25.299714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.305556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1055/2091] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:27:25.306736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:25.308300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.325806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d948a4a-70f5-4e34-8d27-58def3311e0a service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:25.362144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4135d208494495caa74cf2c6a939050', 'request_id': 'req-f0383894-9129-4cb6-948e-1bca390870fa'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.371265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4135d208494495caa74cf2c6a939050', 'request_id': 'req-f0383894-9129-4cb6-948e-1bca390870fa'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.387263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] The token cache is being invalidate because project f4135d208494495caa74cf2c6a939050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:25.387621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.388451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f0383894-9129-4cb6-948e-1bca390870fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.390897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1037/2092] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:25 2022] DELETE /identity/v3/projects/f4135d208494495caa74cf2c6a939050 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:25.402551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:25.403303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.403654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.404005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] PATH_INFO: `/v3/users/da1bee34ddec4286bd6085e6de14d9cc` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.406167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1bee34ddec4286bd6085e6de14d9cc)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:25.409489 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:25.522157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.644408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1bee34ddec4286bd6085e6de14d9cc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.653652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1bee34ddec4286bd6085e6de14d9cc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.666087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1bee34ddec4286bd6085e6de14d9cc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.666823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1bee34ddec4286bd6085e6de14d9cc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.667433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1bee34ddec4286bd6085e6de14d9cc'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:25.778882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1056/2093] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:25 2022] DELETE /identity/v3/users/da1bee34ddec4286bd6085e6de14d9cc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:25.784182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.784465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.784771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.804857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:25.806081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.814148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:25.869234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1038/2094] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:25.879801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:25.881273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:25.942622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a91fff-5340-4842-9418-9bf4fe150bfd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:25.946382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:25.946999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:25.947279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:25.947722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:25.957795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:25.959226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:25.969442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1057/2095] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:26.574769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:26.576907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:26.610794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475ba2a-87d1-44ba-bbef-92e9c3c31885 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:26.620809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:26.621846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:26.622358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:26.622679 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] PATH_INFO: `/v3/projects/d6e67c98bad5401f97ef77e168d8d358` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:26.624884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e67c98bad5401f97ef77e168d8d358)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:26.627831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:26.791542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e67c98bad5401f97ef77e168d8d358', 'request_id': 'req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:26.888828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e67c98bad5401f97ef77e168d8d358', 'request_id': 'req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:26.897232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e67c98bad5401f97ef77e168d8d358', 'request_id': 'req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:26.910338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] The token cache is being invalidate because project d6e67c98bad5401f97ef77e168d8d358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:26.910783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:26.911660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:26.913935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1039/2096] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:26 2022] DELETE /identity/v3/projects/d6e67c98bad5401f97ef77e168d8d358 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:26.918220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:26.931968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e254d-ecdb-4448-b469-b3b9263e2bd0 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:26.991410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:26.992062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:26.992342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:26.992620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] PATH_INFO: `/v3/users/868443c5e16d4a8ab31474884e7b645d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:26.993858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] RBAC: Authorizing `identity:delete_user(user_id=868443c5e16d4a8ab31474884e7b645d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:26.995813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:27.204884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868443c5e16d4a8ab31474884e7b645d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:27.212125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868443c5e16d4a8ab31474884e7b645d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:27.235551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868443c5e16d4a8ab31474884e7b645d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:27.237839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868443c5e16d4a8ab31474884e7b645d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:27.238584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21ae7467-25e2-4e28-a39d-d1451895594c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868443c5e16d4a8ab31474884e7b645d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:27.323029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1058/2097] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:26 2022] DELETE /identity/v3/users/868443c5e16d4a8ab31474884e7b645d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:27:28.301627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:28.303966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.336196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4b74a-8c8a-44a6-b67a-033ba123f17e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:28.350489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:28.404028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:28.404613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.404921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.405202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] PATH_INFO: `/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.406383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe6ff5e8cb14a799d944d82d5a76f8f)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:28.408287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:28.611741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe6ff5e8cb14a799d944d82d5a76f8f', 'request_id': 'req-e0e1e208-ac48-4b20-80a5-60a4be1491a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:28.807084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe6ff5e8cb14a799d944d82d5a76f8f', 'request_id': 'req-e0e1e208-ac48-4b20-80a5-60a4be1491a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:28.815163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe6ff5e8cb14a799d944d82d5a76f8f', 'request_id': 'req-e0e1e208-ac48-4b20-80a5-60a4be1491a1'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:28.831232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] The token cache is being invalidate because project 2fe6ff5e8cb14a799d944d82d5a76f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:28.831558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:28.832180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e0e1e208-ac48-4b20-80a5-60a4be1491a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:28.834130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1040/2098] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:28 2022] DELETE /identity/v3/projects/2fe6ff5e8cb14a799d944d82d5a76f8f => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:28.839549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.840001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.840274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.867356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:28.868573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.883496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:28.940758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1059/2099] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:28.950252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:28.950533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:28.950828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:28.972449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:28.973678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:28.986302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1041/2100] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:28.987501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:28.997746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce52f-8ea0-4bd0-bf52-be0b430dc6b4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.003616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.004223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.004558 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.004857 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.005504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:29.005875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.014127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.020533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1060/2101] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:29.025215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.036196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4de4f-52c3-4d36-941a-b19f8668cd4a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.041523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.042327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.042667 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.047194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.047194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.047194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.153276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1042/2102] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:29.158443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.169183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecff0d7-673d-49b0-91f5-3fc9ebea6a48 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.257238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.257875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.258156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.258431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.259062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.261027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.399758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1061/2103] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/users => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:29.401383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.414810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3857-5222-4f49-83f3-5084b54b01b5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.422130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.422130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.422130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.422434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.423346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.425111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.443913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1043/2104] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:29.445529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.457266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc91c9-a691-489b-83bb-5a558245773d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.462819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.463396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.463796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.464277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.465025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.466968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29f75b-7fbc-42b1-9cf2-a570faa44ed1 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.479849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1062/2105] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:29.484331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.495217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a04e8-a8bc-4268-abd5-a7ffb0a4eff0 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.502964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.503643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.503965 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.504337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] PATH_INFO: `/v3/projects/aba6771fc3504db79bb28aab87856651/users/44aa13af6ff14c25ba5d2eb20c4f62b7/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.521795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba6771fc3504db79bb28aab87856651, user_id=44aa13af6ff14c25ba5d2eb20c4f62b7, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.524830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a37c7c-33cd-4d6b-864d-c2b1bacd5a39 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.599879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.600162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.600435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.621506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] MFA Rules not processed for user `5c8ae69712c14878bd2aab6d33512b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:29.624948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:29.625113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1044/2106] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:29 2022] PUT /identity/v3/projects/aba6771fc3504db79bb28aab87856651/users/44aa13af6ff14c25ba5d2eb20c4f62b7/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:29.638355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.638616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.638890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.669801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] MFA Rules not processed for user `44aa13af6ff14c25ba5d2eb20c4f62b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:29.671503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:29.683281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1063/2107] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:29.711954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:29.747091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1045/2108] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:29.754260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.789341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:29.828290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625c10f-d6ac-4a57-9c41-1c1a227e6887 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:29.833632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:29.834355 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:29.834636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:29.834909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:29.835661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:29.837809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:29.944729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1064/2109] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/projects => generated 407 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:29.946722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:29.959779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf8d44-6771-4ed3-9c54-68782f5023b5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.023922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.024542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.024883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.025223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.025795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.027807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.105136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1046/2110] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:29 2022] POST /identity/v3/users => generated 310 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:30.111008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:30.124776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd574ba-9545-4a5e-ad9b-614df1e74515 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.130351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.131028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.131460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.131804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.132564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.134202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.146448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1065/2111] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:30.151624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:30.164717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccfafb-17b0-4b6b-b3b3-e7f296d4f455 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.170383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.172157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.172157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.172509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] PATH_INFO: `/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5/users/6c58fc3e0f43456cad2fd491322cf17f/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.191086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0e6959b1dc4089a3e16fcbf3b545c5, user_id=6c58fc3e0f43456cad2fd491322cf17f, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.194022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.264642 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1047/2112] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:30 2022] PUT /identity/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5/users/6c58fc3e0f43456cad2fd491322cf17f/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:30.269481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:30.283009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522654c5-addc-4cac-84fc-21c35e8bd873 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.371975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.372827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.373286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.373654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.374833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.377122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138a6f-aef3-46cf-86a3-6a0ce8e6a1e4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.391957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1066/2113] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:30.396244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:30.408820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8d196-0122-4707-82c1-e7925268913e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.413973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.414710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.415217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.415568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] PATH_INFO: `/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5/users/6c58fc3e0f43456cad2fd491322cf17f/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.417668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0e6959b1dc4089a3e16fcbf3b545c5, user_id=6c58fc3e0f43456cad2fd491322cf17f, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.420387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.434104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1048/2114] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:30 2022] PUT /identity/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5/users/6c58fc3e0f43456cad2fd491322cf17f/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:30.444283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.444597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.445640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.488465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] MFA Rules not processed for user `6c58fc3e0f43456cad2fd491322cf17f`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:30.493298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:30.578385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1067/2115] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:30 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:30.591168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:30.593304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:30.674190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2314742-7c65-45a0-86c6-feb7a5be0af0 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:30.678562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:30.679335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:30.679645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:30.679923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:30.690172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:30.691647 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:30.702788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1049/2116] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:30 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:31.102292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:31.104611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:31.118837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3b2e-8a8d-4bf6-8627-9be16b8064fd None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:31.125296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:31.126560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:31.126843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:31.127116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:31.152760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:31.193833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:31.195382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e6361-c4c7-4ee5-aee7-195f003e9bd7 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:31.205496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1068/2117] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:31 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:32.167482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:32.178863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548de3b9-e825-4a79-887a-508676154aba service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:32.184410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:32.185053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:32.185329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:32.185599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:32.189017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:32.226248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:32.267985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:32.269724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:32.281586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1050/2118] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:36.779414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ba7bb-2c95-4687-aaee-545ae87e9030 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:36.779907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ba7bb-2c95-4687-aaee-545ae87e9030 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:36.780120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ba7bb-2c95-4687-aaee-545ae87e9030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:36.801664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fc5ba7bb-2c95-4687-aaee-545ae87e9030 None None] MFA Rules not processed for user `578cd837d9fe442587f8c6c8528334c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:36.802853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fc5ba7bb-2c95-4687-aaee-545ae87e9030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:36.877814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1069/2119] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:42.921810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=17 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:46.864923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=42 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:49.325671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.389132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eec0b1-03b5-42c1-98d4-90a7188dea74 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.394400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.395027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.395308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.395578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] PATH_INFO: `/v3/users/44aa13af6ff14c25ba5d2eb20c4f62b7` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.396708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44aa13af6ff14c25ba5d2eb20c4f62b7)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.398643 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.683183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44aa13af6ff14c25ba5d2eb20c4f62b7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.692733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44aa13af6ff14c25ba5d2eb20c4f62b7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.709409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44aa13af6ff14c25ba5d2eb20c4f62b7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.710359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44aa13af6ff14c25ba5d2eb20c4f62b7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.711098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44aa13af6ff14c25ba5d2eb20c4f62b7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:49.802173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1051/2120] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:49 2022] DELETE /identity/v3/users/44aa13af6ff14c25ba5d2eb20c4f62b7 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:49.807201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.807656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.807794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.829665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:49.830907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:49.899098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1070/2121] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:49.909057 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:49.910617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:49.971862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c0100-efa4-4161-9e1f-8258494e8120 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:49.975449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:49.976058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:49.976335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:49.976610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:49.986664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:49.988190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:49.997966 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1052/2122] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:50.384740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.384740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:50.388456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8785b-db0d-4157-84db-6b0410c430e8 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.393697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.394412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.394690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.394989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] PATH_INFO: `/v3/projects/aba6771fc3504db79bb28aab87856651` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.396287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba6771fc3504db79bb28aab87856651)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.398757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.545322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba6771fc3504db79bb28aab87856651', 'request_id': 'req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.646850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba6771fc3504db79bb28aab87856651', 'request_id': 'req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.653135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba6771fc3504db79bb28aab87856651', 'request_id': 'req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.665283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] The token cache is being invalidate because project aba6771fc3504db79bb28aab87856651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:50.665577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.666230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.667757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1071/2123] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:50 2022] DELETE /identity/v3/projects/aba6771fc3504db79bb28aab87856651 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:50.672791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:50.683171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175011a-ecf3-477d-823d-6f46d5ed90fd service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:50.704269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:50.746618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:50.747618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:50.748049 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:50.750207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] PATH_INFO: `/v3/users/6c58fc3e0f43456cad2fd491322cf17f` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:50.772437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c58fc3e0f43456cad2fd491322cf17f)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:50.772437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:50.986415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c58fc3e0f43456cad2fd491322cf17f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:50.993107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c58fc3e0f43456cad2fd491322cf17f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.004776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c58fc3e0f43456cad2fd491322cf17f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.005447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c58fc3e0f43456cad2fd491322cf17f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.006051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fb385d62-e40c-4a8e-8537-9710e50ee81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c58fc3e0f43456cad2fd491322cf17f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:51.103935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1053/2124] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:27:50 2022] DELETE /identity/v3/users/6c58fc3e0f43456cad2fd491322cf17f => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:51.711794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:51.713284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:51.726820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab93c1-79a6-4351-a1d7-8c84c761d5e3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:51.792220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:51.793106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:51.793365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:51.795826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] PATH_INFO: `/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:51.795826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0e6959b1dc4089a3e16fcbf3b545c5)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:51.798152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:51.935894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:51.955902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0e6959b1dc4089a3e16fcbf3b545c5', 'request_id': 'req-5dc3991a-a94a-4678-ab42-40d7715c385a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.023643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0e6959b1dc4089a3e16fcbf3b545c5', 'request_id': 'req-5dc3991a-a94a-4678-ab42-40d7715c385a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.029628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0e6959b1dc4089a3e16fcbf3b545c5', 'request_id': 'req-5dc3991a-a94a-4678-ab42-40d7715c385a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.042023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] The token cache is being invalidate because project eb0e6959b1dc4089a3e16fcbf3b545c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:27:52.042312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.042984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5dc3991a-a94a-4678-ab42-40d7715c385a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:27:52.044476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1072/2125] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:27:51 2022] DELETE /identity/v3/projects/eb0e6959b1dc4089a3e16fcbf3b545c5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:52.050547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.051037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.051294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.072489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.073729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.137465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1054/2126] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.145382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.145673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.145971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.166463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.167691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.176205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1073/2127] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.180371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.191532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f0a3-d112-4786-8790-f04496138d90 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.196566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.197341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.197505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.197780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.198436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:27:52.198743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.200326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.216032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1055/2128] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:27:52.217339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.227477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fcea0-6616-48a8-ae54-a35ff8406d1f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.232304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.232996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.233275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.234242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.234873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.236828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.347072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1074/2129] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:52.356897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.367845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d901cb-654b-4073-a86c-64d93e85a76d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.394624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:52.425893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.426654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.426936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.427205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.427836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.429815 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.626214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1056/2130] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:52.629718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.641153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189735b3-5a2b-4657-9a1f-c8044a1317d5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.645939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.646546 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.646828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.647096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.647742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.649382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.672708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1075/2131] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:52.676629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.699936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e917719-bee4-425b-9c04-1e91070d5968 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.705795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.706511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.706850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.707187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.707916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.709650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c15719-f185-4f19-95e2-dd40543bd423 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.724141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1057/2132] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:52.725350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:52.737839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f317eb-405b-4e46-a88d-30c054934605 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:52.742460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:52.743084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.743359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.743798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] PATH_INFO: `/v3/projects/f210905da2984274be07bf2427b4b5e4/users/56b1a1799a704867ae1122bc8f525a36/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.758755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f210905da2984274be07bf2427b4b5e4, user_id=56b1a1799a704867ae1122bc8f525a36, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:52.761515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:52.848467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1076/2133] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:52 2022] PUT /identity/v3/projects/f210905da2984274be07bf2427b4b5e4/users/56b1a1799a704867ae1122bc8f525a36/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:52.859797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:52.860077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:52.860350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:52.906042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] MFA Rules not processed for user `56b1a1799a704867ae1122bc8f525a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:52.906042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:52.965515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1058/2134] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:52.969820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.054044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1738b2-e110-4967-b659-af452f5e6e3e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.059260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.060025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.060309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.060585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.061260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.064580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.175244 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1077/2135] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:52 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:27:53.180893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.192997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a97b6-fb86-4c1f-8849-7e7446398401 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.266314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.266911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.267191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.267466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.268239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.275548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.357385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1059/2136] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:53.362437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.375904 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e4cfe-a89c-4192-9327-2b3dd0435376 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.382780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.383513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.384024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.384411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.385283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.387347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.410562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1078/2137] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:53.416919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.429934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28957-606e-42c5-b660-b6bc0dd85d3e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.434922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.437347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.437347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.437347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.437695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.439159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eaf8b6-de6d-4a7c-b3de-8cfc34c50e3d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.450319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1060/2138] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:53.454746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.465974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724638-972f-44a6-be07-1799fb5d61f3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.472789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.473458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.473869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.474153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] PATH_INFO: `/v3/projects/34cba5c1d333405d9112b910c4b6bd19/users/c676a5a2801a4003871a6dc998eaf664/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.492134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cba5c1d333405d9112b910c4b6bd19, user_id=c676a5a2801a4003871a6dc998eaf664, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.495245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.603238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.610964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1079/2139] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:53 2022] PUT /identity/v3/projects/34cba5c1d333405d9112b910c4b6bd19/users/c676a5a2801a4003871a6dc998eaf664/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:53.621180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.621466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.621743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.659763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] MFA Rules not processed for user `c676a5a2801a4003871a6dc998eaf664`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:53.661158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:53.665329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:53.737315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1061/2140] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:27:53.741347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:53.823575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aa435-264b-41de-a1e1-45912224e6d5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:53.828423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:53.829742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:53.830246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:53.830678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:53.831566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:53.833720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:53.992957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1080/2141] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:53.997046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.008440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d356efc9-1bd1-4681-baad-ae4ed57bcdb7 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.072824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.073475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.073763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.074327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.075091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.077793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.191221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1062/2142] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:27:53 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:27:54.195058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.205516 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b39c15-ed59-4c64-9907-f6b52d7961af admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.211128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.211744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.212030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.212304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.213007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.214622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.227896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1081/2143] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:27:54.234139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.245252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12be4-a0f5-4aa1-a338-c15c8b720365 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.250392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.251150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.251427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.251697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] PATH_INFO: `/v3/projects/d50a906f4a7443e99f524e7cc4dd5343/users/df975e21881e441cbc72bcbde949ac99/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.270958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50a906f4a7443e99f524e7cc4dd5343, user_id=df975e21881e441cbc72bcbde949ac99, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.273952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.386992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1063/2144] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:54 2022] PUT /identity/v3/projects/d50a906f4a7443e99f524e7cc4dd5343/users/df975e21881e441cbc72bcbde949ac99/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:54.391962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.402459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c1c3b-a11f-431e-a332-53cc2b7ab4d8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.480163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.480766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.481186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.482050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.482175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.484723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2d4ee-8927-4017-8d27-5a2da712e2dd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.498205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1082/2145] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:27:54.499973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.510922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ad5e1-b36b-43e0-bf3e-b6a2f9387257 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.516968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.517775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.518021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.521952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] PATH_INFO: `/v3/projects/d50a906f4a7443e99f524e7cc4dd5343/users/df975e21881e441cbc72bcbde949ac99/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.521952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50a906f4a7443e99f524e7cc4dd5343, user_id=df975e21881e441cbc72bcbde949ac99, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.523134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.543846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1064/2146] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:27:54 2022] PUT /identity/v3/projects/d50a906f4a7443e99f524e7cc4dd5343/users/df975e21881e441cbc72bcbde949ac99/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:27:54.550786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.552112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.552112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.582581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] MFA Rules not processed for user `df975e21881e441cbc72bcbde949ac99`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:27:54.585386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.659550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:54.668870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1083/2147] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:27:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:27:54.762898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:54.765638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:27:54.827985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494ff1-9e30-4e15-9626-0122a475008b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:54.831729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:54.832463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:54.832897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:54.833173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:54.844311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:54.845893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:54.851425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:27:54.856097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1065/2148] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:27:55.042322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:27:55.070870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84391825-c6ad-4301-8e06-eda5c921018a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:27:55.080644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:27:55.081499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:27:55.081855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:27:55.082200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:27:55.174526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:27:55.179672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adab3d6-a2cd-49a6-97a0-7c5fe0b7f5b0 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:27:55.199678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1084/2149] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:27:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:00.685645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:00.687236 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:00.777822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d4d76-0654-46b0-9256-f1009e440592 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:00.782995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:00.783616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:00.783953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:00.789021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] PATH_INFO: `/v3/users/9404ac21e3204b149482aefa5b992088` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:00.789021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9404ac21e3204b149482aefa5b992088)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:00.789021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:00.881083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:00.881555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:00.881647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:00.901532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:00.902725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:00.965638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1085/2150] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:28:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:01.026487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:01.028056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:01.057135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9404ac21e3204b149482aefa5b992088'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:01.066434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9404ac21e3204b149482aefa5b992088'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:01.082640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9404ac21e3204b149482aefa5b992088'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:01.083456 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9404ac21e3204b149482aefa5b992088'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:01.084241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9404ac21e3204b149482aefa5b992088'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:01.110335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753faf54-ce96-433d-93da-059372fdbcc1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:01.115921 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:01.116531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:01.116848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:01.117120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:01.119396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:01.130499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:01.138122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a609d5-a238-4cd3-bc67-03c4bf1057e1 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:01.142834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1086/2151] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:28:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:28:01.212786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1066/2152] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:00 2022] DELETE /identity/v3/users/9404ac21e3204b149482aefa5b992088 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:01.216795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:01.217149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:01.217492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:01.262251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:01.263490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:01.325268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:01.433366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1087/2153] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:01.451396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:01.453542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:01.592225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c6bb2-25ef-4621-8108-8ea01885de73 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:01.602073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:01.604693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:01.606119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:01.606430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:01.630653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:01.632214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84643ab1-7b28-45b3-a824-0a08feefd3c6 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:01.645134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1067/2154] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:02.325721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:02.327324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:02.362229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1e853-06fd-4c4e-9f25-61a905766674 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:02.382188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:02.383910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:02.384200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:02.388849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] PATH_INFO: `/v3/projects/e68bfeb0ccd84947b9ccc0c41a89d58b` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:02.390318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e68bfeb0ccd84947b9ccc0c41a89d58b)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:02.392294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:02.588963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e68bfeb0ccd84947b9ccc0c41a89d58b', 'request_id': 'req-fb09664c-9a0f-4115-8847-357167e547ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.207339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e68bfeb0ccd84947b9ccc0c41a89d58b', 'request_id': 'req-fb09664c-9a0f-4115-8847-357167e547ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.215177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e68bfeb0ccd84947b9ccc0c41a89d58b', 'request_id': 'req-fb09664c-9a0f-4115-8847-357167e547ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.231645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] The token cache is being invalidate because project e68bfeb0ccd84947b9ccc0c41a89d58b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:03.231938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.232607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.234951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1088/2155] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:02 2022] DELETE /identity/v3/projects/e68bfeb0ccd84947b9ccc0c41a89d58b => generated 0 bytes in 910 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:03.244513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39e7b0-145d-4a67-96b9-d2d9b3a36ba6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.244513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39e7b0-145d-4a67-96b9-d2d9b3a36ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.244513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39e7b0-145d-4a67-96b9-d2d9b3a36ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.267384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fc39e7b0-145d-4a67-96b9-d2d9b3a36ba6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:03.267384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fc39e7b0-145d-4a67-96b9-d2d9b3a36ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.267623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.268122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.293773 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:03.347505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1068/2156] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:03.356658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.359819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.359819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.373479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09664c-9a0f-4115-8847-357167e547ad admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.379696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:03.380539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.380948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.381477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.381926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.382445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] PATH_INFO: `/v3/users/56b1a1799a704867ae1122bc8f525a36` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.384067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b1a1799a704867ae1122bc8f525a36)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.386740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.393561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1069/2157] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:03.400495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.407425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded61c1f-ca02-43fe-a2f3-154879d2033f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.412704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.414998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.414998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.414998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.414998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:03.414998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.416718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.431618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1070/2158] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:28:03.436312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.450023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974ee3-d309-40a4-930c-44b8b0b01ea3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.455771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.456520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.456909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.457262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.458036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.460137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.616464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1071/2159] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:03 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:03.617501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.645390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b1a1799a704867ae1122bc8f525a36'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.655066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:03.660608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b1a1799a704867ae1122bc8f525a36'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.693492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b1a1799a704867ae1122bc8f525a36'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.694566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b1a1799a704867ae1122bc8f525a36'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.695305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-43109937-c434-4019-8aa0-5329c3522818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b1a1799a704867ae1122bc8f525a36'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:03.700801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c95-08a1-4e3b-8384-71564246a021 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.706067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.706682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.706996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.707401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.708152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.710122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.774111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1089/2160] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:03 2022] DELETE /identity/v3/users/56b1a1799a704867ae1122bc8f525a36 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:03.778374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.778664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.778942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.791333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1072/2161] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:03 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:03.799627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.801627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.812150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:03.813576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.816604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d826024-a5c3-4977-bf5a-ea8dcb32ffcf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.892735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1090/2162] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:03.912910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.914615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:03.934752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.935384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.935666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.935958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:03.937076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:03.938823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:03.960515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1073/2163] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:28:03.961419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:03.989079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82cd8-401d-47b7-b6bf-88ec7e4b95d3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:03.996738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:03.998120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:03.998462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:03.998804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.000227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.002158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.004329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:04.020556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1074/2164] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:04.025405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:04.037845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5fe2-014a-46a6-a8f8-02e93373ace2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:04.041569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1cabf-26db-4882-90f4-2b3872337a38 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:04.042647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:04.048547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:04.049339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.049829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.050125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.050242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.050399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.050630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] PATH_INFO: `/v3/projects/cc22daf5edc04e06a3bf88867043a141/users/f3e234a806d24dbe84ce0421bb6e7074/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.065048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.066747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6df424-e4bc-4fe3-8e71-0966612c7b5c service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.095724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1091/2165] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:04.100128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc22daf5edc04e06a3bf88867043a141, user_id=f3e234a806d24dbe84ce0421bb6e7074, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.105133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.273527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.317038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1075/2166] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:04 2022] PUT /identity/v3/projects/cc22daf5edc04e06a3bf88867043a141/users/f3e234a806d24dbe84ce0421bb6e7074/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:04.317565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.317565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.317565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] MFA Rules not processed for user `f3e234a806d24dbe84ce0421bb6e7074`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:04.317565 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:04.444417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1092/2167] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:04.448611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:04.448611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:04.537424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffb0db-bb6c-49fc-bc2c-993461eafb52 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:04.549672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:04.550657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:04.551064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:04.551431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:04.571291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:04.573377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:04.601940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1076/2168] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:05.210657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.211887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:05.220173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:05.333158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f7e4d-5859-4ea5-92a7-ba2c64decc29 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.346970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.347585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.348070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.349911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] PATH_INFO: `/v3/projects/f210905da2984274be07bf2427b4b5e4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.351496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f210905da2984274be07bf2427b4b5e4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.354913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.545187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f210905da2984274be07bf2427b4b5e4', 'request_id': 'req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:05.651688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f210905da2984274be07bf2427b4b5e4', 'request_id': 'req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:05.684500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f210905da2984274be07bf2427b4b5e4', 'request_id': 'req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:05.734586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] The token cache is being invalidate because project f210905da2984274be07bf2427b4b5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:05.737342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:05.739153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:05.745673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1093/2169] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:05 2022] DELETE /identity/v3/projects/f210905da2984274be07bf2427b4b5e4 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:05.753359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.778719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1376ec3-315a-470b-adc2-0afa7ebe708d service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.810626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:05.812043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:05.945447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.946269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.946574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.946850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] PATH_INFO: `/v3/users/c676a5a2801a4003871a6dc998eaf664` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.948002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60fc6c-2b29-4dcd-b615-13c1366e85f5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:05.948221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c676a5a2801a4003871a6dc998eaf664)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.950418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:05.956413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:05.957131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:05.957435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:05.957738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] PATH_INFO: `/v3/users/e971d6fa9eb54088bf7978821653cee1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:05.958910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e971d6fa9eb54088bf7978821653cee1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:05.961122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:06.207754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c676a5a2801a4003871a6dc998eaf664'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.221184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c676a5a2801a4003871a6dc998eaf664'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.243323 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c676a5a2801a4003871a6dc998eaf664'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.244099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c676a5a2801a4003871a6dc998eaf664'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.245093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-06990706-b3f1-4e93-98c6-77a236e98f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c676a5a2801a4003871a6dc998eaf664'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:06.269267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:06.286988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e971d6fa9eb54088bf7978821653cee1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.300056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e971d6fa9eb54088bf7978821653cee1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.320952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e971d6fa9eb54088bf7978821653cee1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.321716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e971d6fa9eb54088bf7978821653cee1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.322411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e971d6fa9eb54088bf7978821653cee1'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:06.347607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1077/2170] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:05 2022] DELETE /identity/v3/users/c676a5a2801a4003871a6dc998eaf664 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:06.432254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1094/2171] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:05 2022] DELETE /identity/v3/users/e971d6fa9eb54088bf7978821653cee1 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:06.442252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:06.442511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:06.442863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:06.548918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:06.552700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:06.690017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1078/2172] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:06.766604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:06.768814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:06.998071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32562e-8fa9-4fa5-91b4-fcb922260261 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:07.020956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:07.021658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:07.021866 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:07.022137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:07.041742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:07.043559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:07.062127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1095/2173] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:07.294838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:07.296973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:07.336645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e1d6-03ad-43b6-987a-796df6a0722c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:07.350063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:07.350740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:07.351022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:07.351299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] PATH_INFO: `/v3/projects/34cba5c1d333405d9112b910c4b6bd19` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:07.353952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cba5c1d333405d9112b910c4b6bd19)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:07.355937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:07.643447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cba5c1d333405d9112b910c4b6bd19', 'request_id': 'req-b3b26467-1960-441e-82e3-c45ee48b41e6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:07.727218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cba5c1d333405d9112b910c4b6bd19', 'request_id': 'req-b3b26467-1960-441e-82e3-c45ee48b41e6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:07.737312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cba5c1d333405d9112b910c4b6bd19', 'request_id': 'req-b3b26467-1960-441e-82e3-c45ee48b41e6'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:07.769029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] The token cache is being invalidate because project 34cba5c1d333405d9112b910c4b6bd19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:07.769402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:07.770677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:07.773419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1079/2174] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:07 2022] DELETE /identity/v3/projects/34cba5c1d333405d9112b910c4b6bd19 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:07.777544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:07.789260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984285e-88c1-4392-98bb-3988cc2cb0d3 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:07.849169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:07.850806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:07.878204 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:07.879144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:07.879620 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:07.880064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] PATH_INFO: `/v3/users/df975e21881e441cbc72bcbde949ac99` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:07.882075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df975e21881e441cbc72bcbde949ac99)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:07.884442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:08.000388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b26467-1960-441e-82e3-c45ee48b41e6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:08.022138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:08.022803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:08.032788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:08.033569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] PATH_INFO: `/v3/projects/f50300d61f164b9aa7c9866eb5d84bde` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:08.040032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50300d61f164b9aa7c9866eb5d84bde)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:08.045479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:08.099926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:08.142668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df975e21881e441cbc72bcbde949ac99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.158766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df975e21881e441cbc72bcbde949ac99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.162099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:08.185629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df975e21881e441cbc72bcbde949ac99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.187699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df975e21881e441cbc72bcbde949ac99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.188481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df975e21881e441cbc72bcbde949ac99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.243883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50300d61f164b9aa7c9866eb5d84bde', 'request_id': 'req-41e27317-cc22-43eb-906e-66e1bcef5ab4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.271438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1096/2175] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:07 2022] DELETE /identity/v3/users/df975e21881e441cbc72bcbde949ac99 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:08.391740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50300d61f164b9aa7c9866eb5d84bde', 'request_id': 'req-41e27317-cc22-43eb-906e-66e1bcef5ab4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.425774 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50300d61f164b9aa7c9866eb5d84bde', 'request_id': 'req-41e27317-cc22-43eb-906e-66e1bcef5ab4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.441308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] The token cache is being invalidate because project f50300d61f164b9aa7c9866eb5d84bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:08.441544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.442576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.445182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1080/2176] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:07 2022] DELETE /identity/v3/projects/f50300d61f164b9aa7c9866eb5d84bde => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:08.449047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:08.451033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:08.556049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247636f1-a77e-4e99-8b32-d705a9ff3487 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:08.565071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:08.565887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:08.566259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:08.566632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] PATH_INFO: `/v3/users/e1559eb1fe7649e58e4d0ad89e2ab486` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:08.567869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1559eb1fe7649e58e4d0ad89e2ab486)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:08.570034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:08.841299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1559eb1fe7649e58e4d0ad89e2ab486'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.848597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1559eb1fe7649e58e4d0ad89e2ab486'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.867692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1559eb1fe7649e58e4d0ad89e2ab486'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.868479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1559eb1fe7649e58e4d0ad89e2ab486'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.869124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d027ae42-a05c-46f9-aa4c-0ddb9ef0cf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1559eb1fe7649e58e4d0ad89e2ab486'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:08.966002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:08.974026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:08.997424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1097/2177] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:08 2022] DELETE /identity/v3/users/e1559eb1fe7649e58e4d0ad89e2ab486 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:09.154090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e27317-cc22-43eb-906e-66e1bcef5ab4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:09.215082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:09.215722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:09.216004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:09.216284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] PATH_INFO: `/v3/projects/d50a906f4a7443e99f524e7cc4dd5343` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:09.217444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50a906f4a7443e99f524e7cc4dd5343)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:09.219976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:09.867816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50a906f4a7443e99f524e7cc4dd5343', 'request_id': 'req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:09.988628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50a906f4a7443e99f524e7cc4dd5343', 'request_id': 'req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:09.997890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50a906f4a7443e99f524e7cc4dd5343', 'request_id': 'req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.010670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] The token cache is being invalidate because project d50a906f4a7443e99f524e7cc4dd5343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:10.011050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.011752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.013290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1081/2178] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:08 2022] DELETE /identity/v3/projects/d50a906f4a7443e99f524e7cc4dd5343 => generated 0 bytes in 1053 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:10.019818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9fe88-a28b-430f-ac7d-1a07dfb45d94 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.020317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9fe88-a28b-430f-ac7d-1a07dfb45d94 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.020412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9fe88-a28b-430f-ac7d-1a07dfb45d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.044377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a8c9fe88-a28b-430f-ac7d-1a07dfb45d94 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:10.051389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a8c9fe88-a28b-430f-ac7d-1a07dfb45d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.051526 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.052077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.151040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1098/2179] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:10.161108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.166325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.166325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.195051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:10.195537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.216028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1099/2180] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:10.222264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.234373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1303e9d-ae2a-4959-89cc-f2e329a4edd9 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.239128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb93b1e-77f7-48da-b1e3-68d7bd25464d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.256696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.256696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.256696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.256696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] PATH_INFO: `/v3/projects/09ade170d87249f680892b2213e8b643` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.257196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ade170d87249f680892b2213e8b643)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.257196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.257374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.257374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.257374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.257374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.257734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:10.257830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.259347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.274999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1100/2181] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:10.279495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.292656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc2ca-5e5f-49ad-9ef2-8f6e906cf629 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.302914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.303575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.303862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.304156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.304872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.306951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.401359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ade170d87249f680892b2213e8b643', 'request_id': 'req-2408f064-b08b-4f8b-9e40-8c8344475e57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.412969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1101/2182] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:10.420944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.489028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31d06a-b6e7-4516-bb37-3e87d3c35162 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.493995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.494705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.494985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.495324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.496099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.498411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.509598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ade170d87249f680892b2213e8b643', 'request_id': 'req-2408f064-b08b-4f8b-9e40-8c8344475e57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.516634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ade170d87249f680892b2213e8b643', 'request_id': 'req-2408f064-b08b-4f8b-9e40-8c8344475e57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.530138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] The token cache is being invalidate because project 09ade170d87249f680892b2213e8b643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:10.531009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.531741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2408f064-b08b-4f8b-9e40-8c8344475e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:10.533465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1082/2183] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:10 2022] DELETE /identity/v3/projects/09ade170d87249f680892b2213e8b643 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:10.538038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bbb32-2da3-466e-9501-f8e4e2696f15 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.542297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bbb32-2da3-466e-9501-f8e4e2696f15 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.542297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bbb32-2da3-466e-9501-f8e4e2696f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.594520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2c2bbb32-2da3-466e-9501-f8e4e2696f15 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:10.595738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2c2bbb32-2da3-466e-9501-f8e4e2696f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.622437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1083/2184] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:10.632043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.632328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.632602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.668940 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:10.670580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.688777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1102/2185] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:10.691996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.696298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1084/2186] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:10.699802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:10.700776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.715376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:10.722244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d025d15-c1d5-4297-a973-64916f2fafb8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.727644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.728260 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.728534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.728838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.731102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f7d1-97eb-4d4d-8f26-7e9bb4a535ac admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.731635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:10.731945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.733712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.745381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.746621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.746910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.749631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.749631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.750697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1085/2187] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:10.753281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.754920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.768734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1103/2188] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:10.770288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.777798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df32b01-f036-4b38-a2c4-6c8d8082c40c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.779258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:10.788131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.788780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.789275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.795535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23a93a-8f4a-4db1-bdb1-4b2d66fdd2eb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.795967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.795967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.795967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.799333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.800213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.800499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.801198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.801861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.803840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.819796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1104/2189] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:10.827751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:10.838779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56661e9e-6357-42ac-8530-b6100cc58178 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:10.844860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:10.845521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:10.845805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:10.846137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] PATH_INFO: `/v3/projects/9e07259e76544a1da77c1a2d34d42132/users/1a540d8eb4784a1a82aa506e840dfcb8/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:10.862095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e07259e76544a1da77c1a2d34d42132, user_id=1a540d8eb4784a1a82aa506e840dfcb8, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:10.864832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab700c-4858-474f-836d-a495a6e2e35f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:10.924035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1086/2190] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:10.928404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.007787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091c8e1-e170-4814-963d-4ee2d319c2f7 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.008034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1105/2191] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:10 2022] PUT /identity/v3/projects/9e07259e76544a1da77c1a2d34d42132/users/1a540d8eb4784a1a82aa506e840dfcb8/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:11.013883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08830d76-6370-4b58-a058-c948c34a45de None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.014170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08830d76-6370-4b58-a058-c948c34a45de None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.014444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08830d76-6370-4b58-a058-c948c34a45de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.042944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-08830d76-6370-4b58-a058-c948c34a45de None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:11.044184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-08830d76-6370-4b58-a058-c948c34a45de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:11.086426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.087273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.087627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.087901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.088693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.091022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.133832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1106/2192] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:11.145834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08830d76-6370-4b58-a058-c948c34a45de None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.147281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-08830d76-6370-4b58-a058-c948c34a45de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:11.184395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1087/2193] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:10 2022] POST /identity/v3/users => generated 284 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:11.189463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.203655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae57be0-9a2b-45fb-aa9d-ff4529925cf6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.210901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.211906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.212524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.212995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.253605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.253605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.253871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08830d76-6370-4b58-a058-c948c34a45de None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.253871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.253871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.253871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.254216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.254310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1088/2194] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:11.254310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.254481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.254481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.254635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5611-32b1-400c-8e45-b08e7d4e242f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.254635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.254635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.254635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.254956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.254956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.254956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37faaf-d900-4027-b2e8-feb00988e524 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.255865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1107/2195] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:11.286333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1089/2196] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:11.290848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.308916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7dc5a-d880-43fd-b53d-99a4ee2bff35 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.319714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.320518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.321021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.321403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] PATH_INFO: `/v3/projects/95a6d973f1574f50bf72e475ab4d7364/users/6efb1b5958d94dbf907a487f23f5efa0/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.356517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a6d973f1574f50bf72e475ab4d7364, user_id=6efb1b5958d94dbf907a487f23f5efa0, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.361327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.471716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.472008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.472289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.474937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1108/2197] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:11 2022] PUT /identity/v3/projects/95a6d973f1574f50bf72e475ab4d7364/users/6efb1b5958d94dbf907a487f23f5efa0/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:11.507428 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] MFA Rules not processed for user `6efb1b5958d94dbf907a487f23f5efa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:11.508658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:11.616618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1090/2198] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:11.621317 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.772395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494495fe-0d89-43ae-962d-0bf14e337758 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:11.784806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:11.785448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:11.785781 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:11.786054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:11.788108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:11.791211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:11.796760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:11.864766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:11.942417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:11.951941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1109/2199] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:11 2022] POST /identity/v3/projects => generated 367 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:11.963707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00d28-50ac-4889-8f09-e3976c462463 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.100952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.104622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.104961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.105241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.109675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.117036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.174144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:12.248712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1091/2200] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:11 2022] POST /identity/v3/users => generated 290 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:12.253125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:12.264431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac8b3-9007-46f1-bdfa-4c3756278322 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.269497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.274552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.274552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.274552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.274552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.274552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.288657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1110/2201] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:12.289724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:12.303030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa8c74-c43d-41bb-8492-2e1500d9014d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.316822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.317218 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.317414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.317696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] PATH_INFO: `/v3/projects/de66b258c8e14c43adba537088096ff5/users/0153e02c39284797aa83144669137b0d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.338424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de66b258c8e14c43adba537088096ff5, user_id=0153e02c39284797aa83144669137b0d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.341585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.457205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1092/2202] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:12 2022] PUT /identity/v3/projects/de66b258c8e14c43adba537088096ff5/users/0153e02c39284797aa83144669137b0d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:12.461451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:12.477472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909f5b0-a3c1-4a26-9883-22674bce759c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.555417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:12.572124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.572124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.572124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.572124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.572742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.576690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b107ed3-3f03-4e87-a64d-378aeb16d26a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.596142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1111/2203] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:12.600778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:12.615054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd25afd-097b-4065-bb8b-1950a05c48b2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.620361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.621430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.621979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.622471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] PATH_INFO: `/v3/projects/de66b258c8e14c43adba537088096ff5/users/0153e02c39284797aa83144669137b0d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.625169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de66b258c8e14c43adba537088096ff5, user_id=0153e02c39284797aa83144669137b0d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.628164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.645556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1093/2204] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:12 2022] PUT /identity/v3/projects/de66b258c8e14c43adba537088096ff5/users/0153e02c39284797aa83144669137b0d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:12.656646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa4fc-47ba-4a74-a36c-a079fbfaefb0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.657030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa4fc-47ba-4a74-a36c-a079fbfaefb0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.657358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa4fc-47ba-4a74-a36c-a079fbfaefb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.727626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1c0aa4fc-47ba-4a74-a36c-a079fbfaefb0 None None] MFA Rules not processed for user `0153e02c39284797aa83144669137b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:12.728922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1c0aa4fc-47ba-4a74-a36c-a079fbfaefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:12.840700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1112/2205] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:12.862369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:12.864699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:12.949133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414c750-8b98-4d23-ae32-a9c92fc09d88 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:12.954483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:12.955280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:12.955711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:12.956167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:12.971730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:12.974023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:12.988901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1094/2206] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:14.050590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.051039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.051313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.062451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.083210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] MFA Rules not processed for user `1a540d8eb4784a1a82aa506e840dfcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:14.084450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:14.096027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f9e3-9eb8-40d4-a2c9-ad21985a0bf9 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:14.099916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:14.100555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.100946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.101223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.164365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1113/2207] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:14.168894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.178574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:14.180528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:14.195309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1095/2208] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:14.276704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5977-a75b-4653-932a-4ba795dc3a06 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:14.282996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:14.283624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.283905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.284173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.284833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:14.287012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:14.295410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:14.436979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1114/2209] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:14 2022] POST /identity/v3/projects => generated 373 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:14.439069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.459640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ccbcb-46ab-43d7-b677-ba38db28e45d service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:14.508002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:14.586211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:14.587224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.587725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.588374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.590004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:14.594692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:14.726388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1096/2210] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:14 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:14.730325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.742713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134527a-2710-4387-ba45-3f40e68c2f4e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:14.747587 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:14.748191 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.748470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.748846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.749665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:14.751253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:14.770486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1115/2211] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:14.774562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.785287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bda96-323a-4f2f-bbe0-44298e39bc98 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:14.789997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:14.790656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:14.790952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:14.791222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] PATH_INFO: `/v3/projects/2f805ae825ff40c39fa30603a5a415e2/users/b4fee9aac7f84e659b7e1d7c1505f9af/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:14.809750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f805ae825ff40c39fa30603a5a415e2, user_id=b4fee9aac7f84e659b7e1d7c1505f9af, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:14.812938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:14.952779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1097/2212] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:14 2022] PUT /identity/v3/projects/2f805ae825ff40c39fa30603a5a415e2/users/b4fee9aac7f84e659b7e1d7c1505f9af/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:14.957133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:14.973676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e544e81-1fa8-4930-a824-c548a6701cdb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:15.035567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:15.036098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:15.036383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:15.036659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:15.037328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:15.038947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7955-9429-4c30-8250-bf10882d50c3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:15.050439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1116/2213] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:28:15.054682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:15.066368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824a910-2c2f-4c9b-a5e2-0dd50ed10e72 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:15.071388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:15.071600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:15.071881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:15.072154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] PATH_INFO: `/v3/projects/2f805ae825ff40c39fa30603a5a415e2/users/b4fee9aac7f84e659b7e1d7c1505f9af/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:15.074101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f805ae825ff40c39fa30603a5a415e2, user_id=b4fee9aac7f84e659b7e1d7c1505f9af, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:15.076712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:15.087715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1098/2214] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:15 2022] PUT /identity/v3/projects/2f805ae825ff40c39fa30603a5a415e2/users/b4fee9aac7f84e659b7e1d7c1505f9af/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:16.727514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:16.728085 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:16.728085 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:16.758243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] MFA Rules not processed for user `b4fee9aac7f84e659b7e1d7c1505f9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:16.759463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:16.840987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1117/2215] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:16.852943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:16.854697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:16.926935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218150b-7793-457c-b196-6b44da2ecf99 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:16.930845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:16.931605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:16.931934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:16.932214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:16.993254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:16.994792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7f8a-9789-4c62-8b55-bea07cd66a53 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:17.005302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1099/2216] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:18.815460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:18.817016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:18.898635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:18.902508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99bf70-0205-403b-ba8a-e1c76ed35aa1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:18.907120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:18.907741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:18.908020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:18.908298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] PATH_INFO: `/v3/users/f3e234a806d24dbe84ce0421bb6e7074` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:18.909465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e234a806d24dbe84ce0421bb6e7074)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:18.911515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:19.118296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e234a806d24dbe84ce0421bb6e7074'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:19.126024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e234a806d24dbe84ce0421bb6e7074'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:19.138211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e234a806d24dbe84ce0421bb6e7074'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:19.138925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e234a806d24dbe84ce0421bb6e7074'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:19.139557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e234a806d24dbe84ce0421bb6e7074'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:19.250697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1118/2217] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:18 2022] DELETE /identity/v3/users/f3e234a806d24dbe84ce0421bb6e7074 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:19.254926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:19.255392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:19.255605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:19.279263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:19.281543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:19.313385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:19.351662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1100/2218] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:19.362522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:19.364016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:19.464763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a18fc-f438-40d3-98c8-74d6a319a2c9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:19.483080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:19.485150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:19.485150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:19.485435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:19.497819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:19.499505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:19.510797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1119/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:19.940308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:19.941705 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:20.000073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fe4d9-03bd-43b6-a2e9-c46a985a756e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.004118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.004975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.005481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.005957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.008413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:20.043229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.044775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:20.067325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccfa74-91c6-4514-8c64-8e4f0666ec7e service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.091656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.097834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d52fe1-fad7-4030-9204-3575f02ac440 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.108015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.109986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.110333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.110814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] PATH_INFO: `/v3/projects/cc22daf5edc04e06a3bf88867043a141` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.112132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc22daf5edc04e06a3bf88867043a141)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.114525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.116866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1101/2220] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:20.239358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc22daf5edc04e06a3bf88867043a141', 'request_id': 'req-21a5fc8b-a5d4-40e3-beed-257b632fffa7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:20.349830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc22daf5edc04e06a3bf88867043a141', 'request_id': 'req-21a5fc8b-a5d4-40e3-beed-257b632fffa7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:20.356068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc22daf5edc04e06a3bf88867043a141', 'request_id': 'req-21a5fc8b-a5d4-40e3-beed-257b632fffa7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:20.367885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] The token cache is being invalidate because project cc22daf5edc04e06a3bf88867043a141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:20.368179 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:20.368848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-21a5fc8b-a5d4-40e3-beed-257b632fffa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:20.370336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1120/2221] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:20 2022] DELETE /identity/v3/projects/cc22daf5edc04e06a3bf88867043a141 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:20.376301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.376802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.376894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.397217 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:20.398396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:20.460047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1102/2222] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:20.472363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.472651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.472957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.480688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:20.500946 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:20.502151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:20.510332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1121/2223] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:20.514583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.524614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2085c-d331-4605-bb4f-c9c2b3b1a77e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.529880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.530594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.530883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.531153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.531809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:20.532119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.533740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.545183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1103/2224] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:20.550253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.559451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b021e-598e-4b36-bde3-7e89751cd29a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.564923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.565877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.566159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.566431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.567093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.569341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.686329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1122/2225] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:20 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:20.689331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.699630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f12b-3a1c-446e-9bf2-60cf59f41099 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.705593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:20.760150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.760794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.761078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.761355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.762040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.763952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.881576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1104/2226] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:20 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:20.885808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.896019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063172c-e669-4564-a11f-2c6134a58006 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.900535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.901158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.901437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.901705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.902394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.903986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.914840 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1123/2227] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:20.919110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.931252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438a9e2-8286-40ca-be60-6d8ba67cc024 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.937025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.937779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.938063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.938343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.939022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.940950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9df586-6684-41f2-b4eb-58a0dc79bc6e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:20.951974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1105/2228] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:28:20.956431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:20.967072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8a221-0a37-4991-a53b-ce60bc6105eb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:20.972222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:20.973012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:20.973612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:20.977169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] PATH_INFO: `/v3/projects/7203c52ae1f946bd9717a554316e4f4c/users/5e3c891f2f3148f4aaba41a14d507cdf/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:20.990431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7203c52ae1f946bd9717a554316e4f4c, user_id=5e3c891f2f3148f4aaba41a14d507cdf, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:20.993117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:21.076194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1124/2229] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:20 2022] PUT /identity/v3/projects/7203c52ae1f946bd9717a554316e4f4c/users/5e3c891f2f3148f4aaba41a14d507cdf/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:21.081787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4edd-4437-4948-84be-cd1991d65b00 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:21.082074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4edd-4437-4948-84be-cd1991d65b00 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:21.082348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4edd-4437-4948-84be-cd1991d65b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:21.103044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-628e4edd-4437-4948-84be-cd1991d65b00 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:21.104344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-628e4edd-4437-4948-84be-cd1991d65b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:21.165215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1106/2230] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:21.175511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:21.177361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:21.235134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeea274-516a-45a1-a618-352ac76a5864 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:21.239398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:21.249813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:21.249813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:21.249813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:21.250084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:21.251476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e45cc-9dfe-4de7-8cd4-d9b18c741159 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:21.261083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1125/2231] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:21.789150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:21.789653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:21.789895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:21.793406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1107/2232] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:28:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:28:21.805498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:21.805786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:21.806255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:21.839812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:21.842003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:21.928162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1126/2233] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:28:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:21.942760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:21.942760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.046719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49925c-bef2-40c1-b36d-6507e38f6b66 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:22.050480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:22.051348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.051640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.051979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.063887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:22.065819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51b8d8-3daa-4459-af70-01fed8eff9cf service glance] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:22.078547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1108/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:22.139131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:22.142022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.162487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12964-7f1f-485c-80b7-59a49374ea76 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:22.167156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:22.167795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.168079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.168359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] PATH_INFO: `/v3/users/6efb1b5958d94dbf907a487f23f5efa0` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.169653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efb1b5958d94dbf907a487f23f5efa0)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:22.171688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:22.300118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:22.357248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a409b-3518-4315-8f03-1d34cebc11d5 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.357537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a409b-3518-4315-8f03-1d34cebc11d5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.357814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a409b-3518-4315-8f03-1d34cebc11d5 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.359320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1109/2235] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:28:22.364342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.364624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.364934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.398845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efb1b5958d94dbf907a487f23f5efa0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:22.401241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:22.402463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.407377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efb1b5958d94dbf907a487f23f5efa0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:22.410156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1110/2236] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:28:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:22.425600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:22.432374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:22.436993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516a473-eb31-434c-bf1e-9b041430c9ee None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:22.437589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efb1b5958d94dbf907a487f23f5efa0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:22.438477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efb1b5958d94dbf907a487f23f5efa0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:22.439564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-45cb5363-165f-44e4-8fdb-26c5bcf303b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efb1b5958d94dbf907a487f23f5efa0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:22.446259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:22.446891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.448463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.448463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.454529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.471028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:22.472546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f2b3f-8fbd-40ee-a9de-b6c8e958481f service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:22.483248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1111/2237] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:22.523021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1127/2238] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:22 2022] DELETE /identity/v3/users/6efb1b5958d94dbf907a487f23f5efa0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:22.527704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.528032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.528272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.549174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:22.550291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.614105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad759-39d4-4716-8aea-68c9064ae9ae None None] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.614618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad759-39d4-4716-8aea-68c9064ae9ae None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.615048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ad759-39d4-4716-8aea-68c9064ae9ae None None] PATH_INFO: `/` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.616495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1128/2239] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:28:22.622411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.623250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.623720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.632603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1112/2240] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:22.642716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:22.644205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.652752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:22.654040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.716549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b8f75-10da-4731-b810-6a50c25f4dee None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:22.722715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:22.723328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.723612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.723876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.734897 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:22.736364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6d495-5958-48fd-bed6-906305ad1498 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:22.739994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1129/2241] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:28:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:22.751545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1113/2242] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:22.753271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:22.755030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:22.766995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845e0d9-2286-4752-b4f4-af3ccb4739c1 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:22.771073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:22.771681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:22.771957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:22.772227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:22.784376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:22.785884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:22.798468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1130/2243] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:23.070777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:23.071321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:23.071321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:23.111541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] MFA Rules not processed for user `5e3c891f2f3148f4aaba41a14d507cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:23.113129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:23.198021 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1114/2244] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:23.211329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:23.212814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:23.317322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c26c-00dd-4e3a-a51b-9640e19e346b service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:23.321358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:23.321941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:23.322221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:23.322494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:23.333325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:23.334887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:23.344515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:23.346905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:23.351252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1131/2245] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:23.369821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be7fc-45f8-4683-8874-2db944789d60 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:23.377413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:23.378243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:23.378682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:23.379437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] PATH_INFO: `/v3/projects/95a6d973f1574f50bf72e475ab4d7364` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:23.380882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a6d973f1574f50bf72e475ab4d7364)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:23.383522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:23.412233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:23.507298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a6d973f1574f50bf72e475ab4d7364', 'request_id': 'req-52a7f828-9b90-483f-a6e9-65b40b91104b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.612433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a6d973f1574f50bf72e475ab4d7364', 'request_id': 'req-52a7f828-9b90-483f-a6e9-65b40b91104b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.618706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a6d973f1574f50bf72e475ab4d7364', 'request_id': 'req-52a7f828-9b90-483f-a6e9-65b40b91104b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.631195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] The token cache is being invalidate because project 95a6d973f1574f50bf72e475ab4d7364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:23.631577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.632243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.634188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1115/2246] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:23 2022] DELETE /identity/v3/projects/95a6d973f1574f50bf72e475ab4d7364 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:28:23.637910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:23.649390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ce34-733d-48da-96bd-bf82d72c879b service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:23.673725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:23.706393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:23.707022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:23.707301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:23.707573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] PATH_INFO: `/v3/users/0153e02c39284797aa83144669137b0d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:23.708767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0153e02c39284797aa83144669137b0d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:23.710718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:23.860802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0153e02c39284797aa83144669137b0d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.867757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0153e02c39284797aa83144669137b0d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.880157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0153e02c39284797aa83144669137b0d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.880891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0153e02c39284797aa83144669137b0d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.881508 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0153e02c39284797aa83144669137b0d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:23.991292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1132/2247] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:23 2022] DELETE /identity/v3/users/0153e02c39284797aa83144669137b0d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:24.036085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:24.038290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:24.163204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7f828-9b90-483f-a6e9-65b40b91104b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:24.227449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:24.228099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:24.228429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:24.228740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:24.230454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:24.242537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:24.244246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644c423-7490-433f-b96d-98a7e6639ace service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:24.255436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1116/2248] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:24.535106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:24.536844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:24.613021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc7e0a-75cf-4946-844e-0c2af4c48327 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:24.617665 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:24.618319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:24.618598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:24.618875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] PATH_INFO: `/v3/projects/de66b258c8e14c43adba537088096ff5` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:24.630409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de66b258c8e14c43adba537088096ff5)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:24.630409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:24.754545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:24.755910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de66b258c8e14c43adba537088096ff5', 'request_id': 'req-e54571d9-7a03-47ed-90de-14dd27ef2b1d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:24.835631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de66b258c8e14c43adba537088096ff5', 'request_id': 'req-e54571d9-7a03-47ed-90de-14dd27ef2b1d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:24.841877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de66b258c8e14c43adba537088096ff5', 'request_id': 'req-e54571d9-7a03-47ed-90de-14dd27ef2b1d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:24.854169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] The token cache is being invalidate because project de66b258c8e14c43adba537088096ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:24.854458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:24.855107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e54571d9-7a03-47ed-90de-14dd27ef2b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:24.860752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1133/2249] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:24 2022] DELETE /identity/v3/projects/de66b258c8e14c43adba537088096ff5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:24.862503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:24.863371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:24.863627 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:24.886267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:24.887470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:24.922652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:24.947763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1117/2250] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:24.956406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:24.956760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:24.957037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:24.977446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:24.978670 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:24.986808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1134/2251] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:24.994742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.005159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c51396-955a-4daa-b2cc-205792cf14cd None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.009751 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.010340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.010837 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.011118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.011792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:25.012093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.013826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.025857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1118/2252] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:25.030968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.040133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670173b-2d8d-4799-b2b5-14d8e4098845 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.044829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.045427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.045708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.046027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.046755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.048646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.210154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1135/2253] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:25.214377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.225092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23577c-8d95-43bb-841e-bcf71740f8fa admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.280280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.281007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.281294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.281573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.282210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.284177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3ba6-0220-496f-9b80-a05ef4dea679 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.375789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1119/2254] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:28:25.380418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.395016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced261c1-1c75-4dac-8d1c-fa3e41740bec admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.400852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.401591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.402115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.402493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.403600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.405780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.407935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840374-ea37-4e5f-b131-4156bd0edcdf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.408427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840374-ea37-4e5f-b131-4156bd0edcdf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.408798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840374-ea37-4e5f-b131-4156bd0edcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.422286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1136/2255] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:28:25.423413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.428949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-91840374-ea37-4e5f-b131-4156bd0edcdf None None] MFA Rules not processed for user `5e3c891f2f3148f4aaba41a14d507cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:25.430169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-91840374-ea37-4e5f-b131-4156bd0edcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:25.435925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f51a10-aae8-4927-9eb9-06a99c21ee16 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.442778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.443618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.444091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.444445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.445613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.447465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.460742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1137/2256] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:25.463042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.473868 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e9b10-16a6-4ca4-9498-e78d8667e293 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.479478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.480283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.483524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.483524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] PATH_INFO: `/v3/projects/f48c50c2ad1e4767b1007ccb2c3aaf60/users/b398823627034485ada7f94e7411ddec/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.496003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48c50c2ad1e4767b1007ccb2c3aaf60, user_id=b398823627034485ada7f94e7411ddec, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.496872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1120/2257] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:25.500453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.574934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1138/2258] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:25 2022] PUT /identity/v3/projects/f48c50c2ad1e4767b1007ccb2c3aaf60/users/b398823627034485ada7f94e7411ddec/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:25.586104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.586493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.586853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.618152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] MFA Rules not processed for user `b398823627034485ada7f94e7411ddec`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:25.619374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:25.712699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:25.726688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1121/2259] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:25.731054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.824981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaab57e-cf7a-4d2a-95b6-135a779db7af admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:25.830364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:25.831208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:25.831592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:25.831963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:25.832745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:25.834801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:25.935965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1139/2260] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:25.941044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:25.951628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e24d0d-782d-48fa-b277-781aa739fd68 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.008051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.008683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.008975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.009253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.009895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.011862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.185087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1122/2261] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:25 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:26.189296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:26.203152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2b04-c11b-4a16-8ea9-373f269879f4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.225277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.226319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.227131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.234758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.234758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.234758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.252242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1140/2262] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:26.256157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:26.270033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f4fc5-1775-4607-82e4-c383c8582a3d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.275533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.276560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.277046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.277732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] PATH_INFO: `/v3/projects/cbbf8cc7acc9423988f65f7282d4839b/users/7db3c219f6c94341bbf04836af0a0021/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.299391 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbf8cc7acc9423988f65f7282d4839b, user_id=7db3c219f6c94341bbf04836af0a0021, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.302804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.425069 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1123/2263] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:26 2022] PUT /identity/v3/projects/cbbf8cc7acc9423988f65f7282d4839b/users/7db3c219f6c94341bbf04836af0a0021/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:26.429218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:26.439975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106402fa-68f7-4845-8351-2ad91151febc admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.475772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:26.500083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.500716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.501044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.501320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.501965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.503583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18993a7-dff7-4837-a5d4-325b0e36fde6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.515855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1141/2264] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:26.533544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:26.551581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b6401-3e61-4b7a-8c62-6262a479cedd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.559844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.561122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.561807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.562359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] PATH_INFO: `/v3/projects/cbbf8cc7acc9423988f65f7282d4839b/users/7db3c219f6c94341bbf04836af0a0021/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.566130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbf8cc7acc9423988f65f7282d4839b, user_id=7db3c219f6c94341bbf04836af0a0021, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.571709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.587108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1124/2265] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:26 2022] PUT /identity/v3/projects/cbbf8cc7acc9423988f65f7282d4839b/users/7db3c219f6c94341bbf04836af0a0021/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:26.598934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.599415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.599553 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.627874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] MFA Rules not processed for user `7db3c219f6c94341bbf04836af0a0021`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:26.629126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:26.702336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1142/2266] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:26.715084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:26.716794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:26.805242 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf838e-9d5b-46ef-ab5b-2ab4c6a14550 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:26.809145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:26.809766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:26.810045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:26.810316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:26.824269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:26.825770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c5f1b-2a50-4847-9590-1088ac3ba370 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:26.840290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1125/2267] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:27.018254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:27.028962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd0223-fb9f-4295-994c-a7a4bfe71abb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:27.034399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:27.035371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:27.035867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:27.036793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:27.097037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:27.097322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:27.099975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:27.100083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:27.100083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:27.110312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1143/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:27.118374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:27.121340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:27.180816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:27.194263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1126/2269] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:28:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:27.206925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:27.208308 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:27.274102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:27.288745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cdd5-a88c-443d-8709-ccd292253178 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:27.294554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:27.294783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:27.294783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:27.294783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:27.307157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:27.308763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559863e-d240-4f4a-af9d-a1548c4af7c0 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:27.317314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1144/2270] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:28:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:28:27.759942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:27.849027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba023fc3-3158-4d99-8e7e-9ed9ea366ff9 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:27.853866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:27.854624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:27.854906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:27.855177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] PATH_INFO: `/v3/users/b398823627034485ada7f94e7411ddec` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:27.856415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b398823627034485ada7f94e7411ddec)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:27.861551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:28.090017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b398823627034485ada7f94e7411ddec'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:28.098953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b398823627034485ada7f94e7411ddec'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:28.114776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b398823627034485ada7f94e7411ddec'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:28.115606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b398823627034485ada7f94e7411ddec'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:28.116886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b398823627034485ada7f94e7411ddec'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:28.218752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1127/2271] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:27 2022] DELETE /identity/v3/users/b398823627034485ada7f94e7411ddec => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:28.223260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:28.223796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:28.223796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:28.244182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:28.245467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:28.309395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1145/2272] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:28.322622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:28.328472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:28.417091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08bacf-429a-48ee-b57f-592038930f26 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:28.420823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:28.421582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:28.421890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:28.422129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:28.434316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:28.435970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:28.447740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1128/2273] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:29.012960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:29.014589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:29.046230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4d2bc-6817-4a8f-bdf9-c0f0d9beafee None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:29.051497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:29.052145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:29.052432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:29.054548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] PATH_INFO: `/v3/projects/f48c50c2ad1e4767b1007ccb2c3aaf60` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:29.054548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48c50c2ad1e4767b1007ccb2c3aaf60)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:29.055817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:29.447938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48c50c2ad1e4767b1007ccb2c3aaf60', 'request_id': 'req-1522a793-49cb-42a0-a1dd-5011b4c00246'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:29.503634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48c50c2ad1e4767b1007ccb2c3aaf60', 'request_id': 'req-1522a793-49cb-42a0-a1dd-5011b4c00246'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:29.511106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48c50c2ad1e4767b1007ccb2c3aaf60', 'request_id': 'req-1522a793-49cb-42a0-a1dd-5011b4c00246'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:29.526674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] The token cache is being invalidate because project f48c50c2ad1e4767b1007ccb2c3aaf60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:29.527239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:29.528061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:29.529923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1146/2274] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:29 2022] DELETE /identity/v3/projects/f48c50c2ad1e4767b1007ccb2c3aaf60 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:29.534695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:29.550259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f77dbd-3e49-4294-8a25-f7c41f5b30b3 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:29.561108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:29.648864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:29.649684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:29.650151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:29.650551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] PATH_INFO: `/v3/users/7db3c219f6c94341bbf04836af0a0021` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:29.652100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db3c219f6c94341bbf04836af0a0021)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:29.654269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:30.364807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db3c219f6c94341bbf04836af0a0021'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:30.372640 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db3c219f6c94341bbf04836af0a0021'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:30.392868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db3c219f6c94341bbf04836af0a0021'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:30.393936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db3c219f6c94341bbf04836af0a0021'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:30.394649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f40f67f7-6632-46a9-8f39-2390c944f871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db3c219f6c94341bbf04836af0a0021'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:30.625373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1129/2275] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:29 2022] DELETE /identity/v3/users/7db3c219f6c94341bbf04836af0a0021 => generated 0 bytes in 1054 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:31.199112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:31.201674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:31.220120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522a793-49cb-42a0-a1dd-5011b4c00246 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:31.306245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:31.306879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:31.307159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:31.307430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] PATH_INFO: `/v3/projects/cbbf8cc7acc9423988f65f7282d4839b` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:31.308771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbf8cc7acc9423988f65f7282d4839b)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:31.310716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:31.327832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:31.446455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbf8cc7acc9423988f65f7282d4839b', 'request_id': 'req-800ccc63-fc97-4f15-be4f-f838656b5802'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:31.527486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbf8cc7acc9423988f65f7282d4839b', 'request_id': 'req-800ccc63-fc97-4f15-be4f-f838656b5802'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:31.535448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbf8cc7acc9423988f65f7282d4839b', 'request_id': 'req-800ccc63-fc97-4f15-be4f-f838656b5802'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:31.556241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] The token cache is being invalidate because project cbbf8cc7acc9423988f65f7282d4839b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:31.556596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:31.557383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-800ccc63-fc97-4f15-be4f-f838656b5802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:31.559046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1147/2276] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:31 2022] DELETE /identity/v3/projects/cbbf8cc7acc9423988f65f7282d4839b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:31.564998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:31.569967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:31.569967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:31.604764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:31.607893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:31.692719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1130/2277] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:28:31.695867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:31.695867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:31.696073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:31.717900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:31.720267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:31.730016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1148/2278] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:31.732899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:31.744152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248cc53-b205-4652-adba-ca0dcc0cd1f5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:31.751357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:31.752110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:31.752398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:31.752692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:31.754162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:31.755015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:31.756072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:31.775554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1131/2279] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:31.776703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:31.793332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1e52c-f1b4-4c14-be5c-84fafde1226e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:31.813687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:31.814362 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:31.814680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:31.814981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:31.818699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:31.818699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:31.948942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1149/2280] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:31 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:31.957511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:31.968827 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8c112-f213-4198-8abe-08ea4ce3e3c8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:31.988982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:32.031612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.032253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.032536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.032877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.033570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.035610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:32.183433 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1132/2281] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:31 2022] POST /identity/v3/users => generated 281 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:32.185144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:32.198463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838e018-6f1c-4a6c-b569-c6a85ab81ed5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:32.203972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.204640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.204640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.204640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.205287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.207216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:32.222162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1150/2282] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:32.223672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:32.235270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e23319-7855-44bf-a9c4-83055e336b06 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:32.240473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.241131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.241415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.241682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.242497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.244089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c289602-41ad-49bd-88e9-54ffffe96099 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:32.255429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1133/2283] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:32.260498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:32.272606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5be340-a2e4-41d0-9124-c0e2daa20128 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:32.277487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.278222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.278575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.278924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] PATH_INFO: `/v3/projects/6043a15f1ab74da880a5d5d6406a4fad/users/60e5415736924593b6a251eb99f39409/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.298606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6043a15f1ab74da880a5d5d6406a4fad, user_id=60e5415736924593b6a251eb99f39409, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.301444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:32.414507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1151/2284] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:32 2022] PUT /identity/v3/projects/6043a15f1ab74da880a5d5d6406a4fad/users/60e5415736924593b6a251eb99f39409/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:32.425454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.425738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.426012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.468898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] MFA Rules not processed for user `60e5415736924593b6a251eb99f39409`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:32.470114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:32.551364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1134/2285] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:32.667638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:32.669397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:32.731851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc3e95-e8cb-400f-ba86-709990f76bff admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:32.735902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.736523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.736836 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.737201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.746075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.775591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2ae6b-86cc-4618-ac62-a682799b8453 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:32.775591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1152/2286] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:32.782754 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:32.904883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c47c6-1371-4110-abb6-e8569ba10822 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:32.915006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:32.915659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:32.915936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:32.916299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] PATH_INFO: `/v3/users/60e5415736924593b6a251eb99f39409` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:32.917478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e5415736924593b6a251eb99f39409)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:32.919639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:33.122048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:33.145393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e5415736924593b6a251eb99f39409'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:33.154169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e5415736924593b6a251eb99f39409'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:33.166847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e5415736924593b6a251eb99f39409'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:33.169661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e5415736924593b6a251eb99f39409'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:33.169661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e5415736924593b6a251eb99f39409'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:33.255499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1135/2287] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:32 2022] DELETE /identity/v3/users/60e5415736924593b6a251eb99f39409 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:33.259716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:33.260267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:33.260389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:33.284396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:33.294027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:33.325364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:33.361406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1153/2288] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:33.373043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:33.375564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:33.441422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8131b4c-8412-4a52-8129-750f84c78ebe admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:33.455054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:33.455829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:33.456065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:33.456383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:33.472104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:33.473896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad8a9e-54eb-4d64-9a77-5ab35e5d0860 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:33.485269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1136/2289] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:33.989838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:33.991575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:34.008913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb8acc-51b3-4e17-a5a3-e4f2df40da37 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.019843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.020475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.020870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.021122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] PATH_INFO: `/v3/projects/6043a15f1ab74da880a5d5d6406a4fad` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.022550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6043a15f1ab74da880a5d5d6406a4fad)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.024757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.219281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6043a15f1ab74da880a5d5d6406a4fad', 'request_id': 'req-ae63cda4-22ad-484a-8b03-e5e371f4f089'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:34.332842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6043a15f1ab74da880a5d5d6406a4fad', 'request_id': 'req-ae63cda4-22ad-484a-8b03-e5e371f4f089'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:34.341534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6043a15f1ab74da880a5d5d6406a4fad', 'request_id': 'req-ae63cda4-22ad-484a-8b03-e5e371f4f089'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:34.354357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] The token cache is being invalidate because project 6043a15f1ab74da880a5d5d6406a4fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:34.355052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:34.358359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ae63cda4-22ad-484a-8b03-e5e371f4f089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:34.360496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1154/2290] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:33 2022] DELETE /identity/v3/projects/6043a15f1ab74da880a5d5d6406a4fad => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:34.371752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.372392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.372392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.395806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:34.402444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:34.471358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1137/2291] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:34.480638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.480945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.481221 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.502120 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:34.503348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:34.511456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1155/2292] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:34.516491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.531122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ef38b-104d-4269-a3e8-270c29e68e11 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.536379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.537032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.537314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.537699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.538364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:34.538666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.540265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.554251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1138/2293] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:28:34.559574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.571865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d0fb-ad2b-485a-a0c8-3a6dfc1df861 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.577621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.578878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.579218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.579567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.581644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.583624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.724677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.727375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1156/2294] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:34 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:34.736063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd8e33-9de0-4c64-9589-7e6e1444648c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.783138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:34.811082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.812561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.812983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.813390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.814027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.816477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.932791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1139/2295] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:34 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:34.933669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.948263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33411b6a-904a-4937-bf73-d8f0c21f940e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.953153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.953926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.954127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.954461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.955228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.956928 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:34.968438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1157/2296] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:34.972893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:34.983001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca420a16-ba1a-4c6a-8c30-515165d5ae2b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:34.989211 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:34.992741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:34.992741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:34.992741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:34.992741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:34.993254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4ea5d-36da-4559-aa7a-38df8226f67f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:35.010329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1140/2297] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:35.012023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:35.032635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a461-ae4b-4aca-af23-0f9a9dd4ee68 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:35.038517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:35.041492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:35.041492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:35.041492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] PATH_INFO: `/v3/projects/0d01536a582e4b82b065bae239243f86/users/acacfb61a39b43518de0ed4cf4c348cc/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:35.065187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d01536a582e4b82b065bae239243f86, user_id=acacfb61a39b43518de0ed4cf4c348cc, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:35.068320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:35.143102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1158/2298] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:35 2022] PUT /identity/v3/projects/0d01536a582e4b82b065bae239243f86/users/acacfb61a39b43518de0ed4cf4c348cc/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:35.147626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:35.147938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:35.148213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:35.181772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] MFA Rules not processed for user `acacfb61a39b43518de0ed4cf4c348cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:35.183185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:35.263523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1141/2299] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:35.277939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:35.280845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:35.422691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba568cf7-162f-4da2-b742-f1152c93d2f0 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:35.422691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:35.422691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:35.422691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:35.423635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:35.429339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:35.430883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:35.436732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:35.443070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1159/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:38.579728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:38.585621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:38.681515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32a71e-190d-4f93-af57-04c146a28f1f None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:38.696576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:38.697301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:38.697598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:38.697881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] PATH_INFO: `/v3/users/1a540d8eb4784a1a82aa506e840dfcb8` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:38.699050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a540d8eb4784a1a82aa506e840dfcb8)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:38.705887 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:39.128419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a540d8eb4784a1a82aa506e840dfcb8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:39.130162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:39.143405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a540d8eb4784a1a82aa506e840dfcb8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:39.162938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a540d8eb4784a1a82aa506e840dfcb8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:39.166723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a540d8eb4784a1a82aa506e840dfcb8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:39.166723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a540d8eb4784a1a82aa506e840dfcb8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:39.292817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1142/2301] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:38 2022] DELETE /identity/v3/users/1a540d8eb4784a1a82aa506e840dfcb8 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:39.695197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:39.708448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533e99f-8e12-4555-adfb-4d7670c16446 service glance] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:39.784439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:39.785188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:39.785530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:39.785838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] PATH_INFO: `/v3/projects/9e07259e76544a1da77c1a2d34d42132` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:39.787057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e07259e76544a1da77c1a2d34d42132)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:39.789046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:39.924934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e07259e76544a1da77c1a2d34d42132', 'request_id': 'req-5864236a-4f34-42d9-b288-c47597782afe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.006797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e07259e76544a1da77c1a2d34d42132', 'request_id': 'req-5864236a-4f34-42d9-b288-c47597782afe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.013291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e07259e76544a1da77c1a2d34d42132', 'request_id': 'req-5864236a-4f34-42d9-b288-c47597782afe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.025413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] The token cache is being invalidate because project 9e07259e76544a1da77c1a2d34d42132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:40.025704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.026360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.028011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1160/2302] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:39 2022] DELETE /identity/v3/projects/9e07259e76544a1da77c1a2d34d42132 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:40.032237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:40.033712 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:40.049153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904fbed-0be9-4786-aff1-01c308abf35d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:40.120143 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:40.120848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:40.121127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:40.121402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] PATH_INFO: `/v3/users/b4fee9aac7f84e659b7e1d7c1505f9af` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:40.122553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fee9aac7f84e659b7e1d7c1505f9af)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:40.125511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:40.301818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fee9aac7f84e659b7e1d7c1505f9af'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.311536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fee9aac7f84e659b7e1d7c1505f9af'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.325181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fee9aac7f84e659b7e1d7c1505f9af'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.325930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fee9aac7f84e659b7e1d7c1505f9af'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.326755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-217edb76-2c57-46c3-9c98-722100ea085c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fee9aac7f84e659b7e1d7c1505f9af'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:40.394659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1143/2303] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:28:40 2022] DELETE /identity/v3/users/b4fee9aac7f84e659b7e1d7c1505f9af => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:40.791172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:40.792788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:40.809208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864236a-4f34-42d9-b288-c47597782afe admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:40.879909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:40.881144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:40.881645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:40.882223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] PATH_INFO: `/v3/projects/2f805ae825ff40c39fa30603a5a415e2` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:40.885571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f805ae825ff40c39fa30603a5a415e2)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:40.887852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.097919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:41.106254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f805ae825ff40c39fa30603a5a415e2', 'request_id': 'req-9cbe1f08-b556-4985-8df4-6ec697b422d7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:41.190002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f805ae825ff40c39fa30603a5a415e2', 'request_id': 'req-9cbe1f08-b556-4985-8df4-6ec697b422d7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:41.198496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f805ae825ff40c39fa30603a5a415e2', 'request_id': 'req-9cbe1f08-b556-4985-8df4-6ec697b422d7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:41.219910 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] The token cache is being invalidate because project 2f805ae825ff40c39fa30603a5a415e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:28:41.220404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:41.221632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9cbe1f08-b556-4985-8df4-6ec697b422d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:28:41.227172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1161/2304] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:28:40 2022] DELETE /identity/v3/projects/2f805ae825ff40c39fa30603a5a415e2 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:41.229008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.229495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.229598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.257216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:41.258760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:41.327335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:41.348298 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1144/2305] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:41.356397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.356745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.357035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.380962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:41.382289 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:41.390439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1162/2306] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:41.394848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.404654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28055f6-1084-4263-9546-014f17ce7b8a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.409652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.410359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.410728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.411063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.414694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:28:41.414694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.414694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.428615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1145/2307] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:28:41.432199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.443706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f51362-6b84-4953-9330-5cd93e83d5f5 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.448734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.449341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.449617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.449886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.450512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.452588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.553067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1163/2308] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:41 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:41.557319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.570908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4ea5-8e25-4a69-b159-5abc59214c26 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.630802 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.631447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.631728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.632071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.632747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.634686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.740173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1146/2309] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:41 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:41.745133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.755952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b64-e769-4572-8176-132ea44057fa admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.761640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.762338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.762742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.763302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.763951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.765636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.779134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1164/2310] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:41.780838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.790781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295dcf9-edef-4579-b8bf-73457918c530 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.795440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.796585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.796898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.797168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.797817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.799422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2f2f0-4614-47de-873d-eb5ff6c57c98 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.810226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1147/2311] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:28:41.814880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:41.826524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86158a8f-fad0-4252-b56c-7f52c5e4671c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:41.834056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:41.834997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.835373 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.835697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] PATH_INFO: `/v3/projects/fa4edbfca7b749ff9be3268fa6b81264/users/8be4cb4f1c594f42825eb40985a29cac/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.851430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4edbfca7b749ff9be3268fa6b81264, user_id=8be4cb4f1c594f42825eb40985a29cac, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:41.854229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:41.945022 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1165/2312] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:41 2022] PUT /identity/v3/projects/fa4edbfca7b749ff9be3268fa6b81264/users/8be4cb4f1c594f42825eb40985a29cac/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:41.951547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574107b9-e96c-4ded-ae3d-c3f99804e7c8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:41.951836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574107b9-e96c-4ded-ae3d-c3f99804e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:41.952798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574107b9-e96c-4ded-ae3d-c3f99804e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:41.972311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-574107b9-e96c-4ded-ae3d-c3f99804e7c8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:41.976473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-574107b9-e96c-4ded-ae3d-c3f99804e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:42.034314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1148/2313] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:42.044255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:42.046706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:42.106449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7a9c-b08a-42ce-9d01-c01c91083660 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:42.111000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:42.111632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:42.111920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:42.112201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:42.122488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:42.124500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:42.135049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1166/2314] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:28:48.273744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.274380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.275499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.318568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] MFA Rules not processed for user `8be4cb4f1c594f42825eb40985a29cac`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:48.320620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:48.405092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:48.439856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1149/2315] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:48.444703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.456979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d7a55-825f-4869-9fc6-edfcc0faa0bc service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.464709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.464709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.465795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.466079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.466721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.469691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.555574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1167/2316] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:28:48 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:48.557068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.569682 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db755030-21fa-4ce1-aa01-cc04da6f587c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.658317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.658963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.659241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.659513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.660220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.662529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.767894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1150/2317] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:28:48 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:28:48.771843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.793092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31d58-e93c-428e-8a3e-1519cff4e2c8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.793092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.793092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.793092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.793675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.793675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.793675 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.809502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.810214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1168/2318] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:48.820883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293613e-8ca5-458a-9e36-dd65701e3f55 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:48.826715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:48.827768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:48.828116 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:48.828804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] PATH_INFO: `/v3/projects/7fc76c9217b04e3ebdc10482112ff802/users/a2690d27ae8e4125a9640e9bf45e202d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:48.847860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc76c9217b04e3ebdc10482112ff802, user_id=a2690d27ae8e4125a9640e9bf45e202d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:48.851171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:48.912208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1151/2319] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:48 2022] PUT /identity/v3/projects/7fc76c9217b04e3ebdc10482112ff802/users/a2690d27ae8e4125a9640e9bf45e202d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:48.917425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:48.937513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f66d-6002-4f42-a405-72677f812064 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:49.010593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:49.062970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:49.063832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:49.068213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:49.068213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:49.068213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:49.068213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982b822-24c6-4425-a233-24aeedb19307 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:49.090101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1169/2320] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:28:49.095232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:49.112251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b57d-217c-4d0f-8728-f4a3308966a6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:49.117670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:49.118571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:49.119915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:49.119915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] PATH_INFO: `/v3/projects/7fc76c9217b04e3ebdc10482112ff802/users/a2690d27ae8e4125a9640e9bf45e202d/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:49.121894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc76c9217b04e3ebdc10482112ff802, user_id=a2690d27ae8e4125a9640e9bf45e202d, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:49.124946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:49.141278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1152/2321] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:28:49 2022] PUT /identity/v3/projects/7fc76c9217b04e3ebdc10482112ff802/users/a2690d27ae8e4125a9640e9bf45e202d/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:28:51.826053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89b88b-2b29-4dbc-9dbe-6fdfe3feb54a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:51.826053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89b88b-2b29-4dbc-9dbe-6fdfe3feb54a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:51.826053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89b88b-2b29-4dbc-9dbe-6fdfe3feb54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:51.850157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bb89b88b-2b29-4dbc-9dbe-6fdfe3feb54a None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:51.851706 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bb89b88b-2b29-4dbc-9dbe-6fdfe3feb54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:51.951110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1170/2322] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:28:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:51.952941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:51.957577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:52.175201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264869d1-e983-482d-92e3-6888cb5a2d1d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:52.187121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:52.188510 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:52.189115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:52.190084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:52.204898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:52.206709 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:52.219258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1153/2323] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:28:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:28:55.975287 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54725ad-1266-4a4a-b911-a43100638432 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:55.975767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54725ad-1266-4a4a-b911-a43100638432 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:55.975863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54725ad-1266-4a4a-b911-a43100638432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:56.010924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f54725ad-1266-4a4a-b911-a43100638432 None None] MFA Rules not processed for user `a2690d27ae8e4125a9640e9bf45e202d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:56.012286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f54725ad-1266-4a4a-b911-a43100638432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:56.153563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1171/2324] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:28:56.169550 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:28:56.171085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:56.201332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:28:56.258846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede530c-9f9b-488f-9f68-528c36c11f9f service swift] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:28:56.266450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:28:56.267357 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:56.267932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:56.268523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:56.363294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:28:56.366906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:28:56.377330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1154/2325] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:28:59.310714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83597d2-557b-489c-9ba4-933a5b1d615c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:28:59.311198 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83597d2-557b-489c-9ba4-933a5b1d615c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:28:59.311336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83597d2-557b-489c-9ba4-933a5b1d615c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:28:59.331811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a83597d2-557b-489c-9ba4-933a5b1d615c None None] MFA Rules not processed for user `8be4cb4f1c594f42825eb40985a29cac`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:28:59.333170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a83597d2-557b-489c-9ba4-933a5b1d615c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:28:59.340180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1172/2326] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:28:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:06.372442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=46 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:09.325734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=79 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:15.129668 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:15.131163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:15.217276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54c9c-2739-4ed0-baec-d24dd225a349 service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:15.231752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:15.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:15.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:15.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] PATH_INFO: `/v3/users/acacfb61a39b43518de0ed4cf4c348cc` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:15.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acacfb61a39b43518de0ed4cf4c348cc)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:15.255312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:15.490636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acacfb61a39b43518de0ed4cf4c348cc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:15.505660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acacfb61a39b43518de0ed4cf4c348cc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:15.523936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acacfb61a39b43518de0ed4cf4c348cc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:15.527418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acacfb61a39b43518de0ed4cf4c348cc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:15.527418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acacfb61a39b43518de0ed4cf4c348cc'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:15.615115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1155/2327] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:29:15 2022] DELETE /identity/v3/users/acacfb61a39b43518de0ed4cf4c348cc => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:15.619424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:15.619902 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:15.619996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:15.642773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:15.645554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:15.719194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1173/2328] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:15.729747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:15.731595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:15.801778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d5ae1-0ede-457a-bf60-3c844dc52a12 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:15.805683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:15.812113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:15.812113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:15.812113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:15.817389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:15.818822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3183a9b-3114-4fd7-8408-6a82ccc6e4b9 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:15.829060 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1156/2329] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:16.289290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:16.290837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:16.304329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a482e9-3f55-44c9-9321-0912e574b147 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:16.310567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:16.311424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:16.311865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:16.312303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] PATH_INFO: `/v3/projects/0d01536a582e4b82b065bae239243f86` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:16.314481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d01536a582e4b82b065bae239243f86)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:16.316572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:16.476496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d01536a582e4b82b065bae239243f86', 'request_id': 'req-194a0b41-b499-40fd-969d-363d45ffebf7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:16.555106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d01536a582e4b82b065bae239243f86', 'request_id': 'req-194a0b41-b499-40fd-969d-363d45ffebf7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:16.563063 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d01536a582e4b82b065bae239243f86', 'request_id': 'req-194a0b41-b499-40fd-969d-363d45ffebf7'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:16.578995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] The token cache is being invalidate because project 0d01536a582e4b82b065bae239243f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:29:16.578995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:16.579708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-194a0b41-b499-40fd-969d-363d45ffebf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:29:16.581262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1174/2330] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:29:16 2022] DELETE /identity/v3/projects/0d01536a582e4b82b065bae239243f86 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:29:18.318794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:18.319289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:18.319404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:18.339500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] MFA Rules not processed for user `5c8ae69712c14878bd2aab6d33512b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:18.340737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:18.358981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:18.401373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1157/2331] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:29:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:20.569890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3537da-a286-457a-bc5c-715a7581f7f7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:20.570367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3537da-a286-457a-bc5c-715a7581f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:20.570459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3537da-a286-457a-bc5c-715a7581f7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:20.593220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5b3537da-a286-457a-bc5c-715a7581f7f7 None None] MFA Rules not processed for user `578cd837d9fe442587f8c6c8528334c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:20.593220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5b3537da-a286-457a-bc5c-715a7581f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:20.647454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1175/2332] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:29:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:29:28.898115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=15 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:30.640484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:29:55.379584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:29:55.382007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:55.440171 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec58b17-5cd8-4fea-b2ca-bec05bcea0f0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:29:55.444105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:29:55.444735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:55.445118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:55.445431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:55.447167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:55.516971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:29:55.518407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71274-4b87-48f9-831e-686cad720812 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:29:55.548557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1158/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:29:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:29:59.687368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9005dd-09f7-4169-8b2e-2262a4e06266 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:29:59.687838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9005dd-09f7-4169-8b2e-2262a4e06266 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:29:59.687922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9005dd-09f7-4169-8b2e-2262a4e06266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:29:59.707500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1c9005dd-09f7-4169-8b2e-2262a4e06266 None None] MFA Rules not processed for user `5e3c891f2f3148f4aaba41a14d507cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:29:59.708761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1c9005dd-09f7-4169-8b2e-2262a4e06266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:29:59.763782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1176/2334] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:29:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:05.525420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=50 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:07.952645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:07.953338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:07.953338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:07.972927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] MFA Rules not processed for user `d451fb2f26ce4f50b010fb2a387c4bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:07.974262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:08.035258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1159/2335] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:30:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:30:09.758411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=25 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:16.930452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:16.931043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:16.931187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:16.951574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] MFA Rules not processed for user `eb4c92edcb024e32a549650eb85f6952`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:16.952827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:17.008737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1177/2336] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:30:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:17.130246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:17.140838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aada1-e994-4c2a-9ed7-8fc6d9b749f5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:17.144540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:17.145179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:17.145529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:17.145807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:17.147564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:17.228086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:17.230789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:17.242011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1160/2337] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:20.882453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:20.892503 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6fd71-1be3-4e33-9359-bda600f4cc54 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:20.896206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:20.896838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:20.897124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:20.897401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:20.899140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:20.913293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:20.914817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:20.927372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1178/2338] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:27.237100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=59 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:28.000253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:28.001692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:28.016961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ede93-4849-46e9-b2cf-a79e1fa61110 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:28.022468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:28.023101 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:28.023387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:28.023664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:28.025381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:28.035635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:28.037398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:28.047661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1161/2339] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:28.325695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:28.337548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783c221-c3ef-40ad-a286-dc85ad8dfebc service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:28.343791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:28.344434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:28.344750 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:28.345030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:28.347037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:28.359184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:28.361989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:28.370264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1179/2340] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:30.546368 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:30.561604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c9bcb-8deb-4933-8757-2aeded4016de service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:30.570459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:30.571139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:30.571371 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:30.571648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:30.574564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:30.586222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:30.587998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:30.597894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1162/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:30:38.365788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=37 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:40.593338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=12 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:42.984887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:42.984887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:42.994028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e684f-aab0-45b5-9347-9d4decd87231 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:42.997764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:42.998430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:42.998715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:42.998984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:43.001151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:43.011787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:43.013280 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:43.021452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1180/2342] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:53.017944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:53.637434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:53.640054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:53.660144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58b4b-8cc0-4497-95f8-641d6fd7a5cb service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:53.669628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:53.670730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:53.671262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:53.672233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:53.674694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:53.698807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:53.701580 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:53.718611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1163/2343] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:54.298239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:54.307710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc176-0794-4b42-aa5a-4cc166d9eac4 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:54.312472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:54.313732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:54.314039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:54.314381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:54.317168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:54.384738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:54.386751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:54.397930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1181/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:58.554848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:58.556405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:58.641726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9213f0c-dd17-4766-a2bd-a408e9d0c60c service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:58.647031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:58.649811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:58.649811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:58.649811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:58.650503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:58.662775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:58.664380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:58.675778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1164/2345] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:30:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:30:58.734648 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:58.736843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:58.753307 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a290e-1ac7-41bf-887a-00dc30fed0db service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:58.758658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:58.759329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:58.759667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:58.759996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] PATH_INFO: `/v3/users/5c8ae69712c14878bd2aab6d33512b7a` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:58.761376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8ae69712c14878bd2aab6d33512b7a)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:58.763333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:59.021651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8ae69712c14878bd2aab6d33512b7a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.030229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8ae69712c14878bd2aab6d33512b7a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.044980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8ae69712c14878bd2aab6d33512b7a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.045853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8ae69712c14878bd2aab6d33512b7a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.046811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e6847d0-b450-4a9b-a287-c01064656cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8ae69712c14878bd2aab6d33512b7a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.122858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1182/2346] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:30:58 2022] DELETE /identity/v3/users/5c8ae69712c14878bd2aab6d33512b7a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:59.437068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:59.450935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc872c-7f33-4837-aab7-ade4a2896bb0 service placement] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:59.516352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:59.517016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.517330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.517568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] PATH_INFO: `/v3/projects/8af7d2bbc6d94273bd39c81423d8cebe` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.518689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af7d2bbc6d94273bd39c81423d8cebe)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:59.520605 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:59.666308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af7d2bbc6d94273bd39c81423d8cebe', 'request_id': 'req-96785fdc-b78c-46cf-8457-183570759057'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.760435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af7d2bbc6d94273bd39c81423d8cebe', 'request_id': 'req-96785fdc-b78c-46cf-8457-183570759057'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.766912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af7d2bbc6d94273bd39c81423d8cebe', 'request_id': 'req-96785fdc-b78c-46cf-8457-183570759057'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.779867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] The token cache is being invalidate because project 8af7d2bbc6d94273bd39c81423d8cebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:30:59.780188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.780903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-96785fdc-b78c-46cf-8457-183570759057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:30:59.782519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1165/2347] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:30:59 2022] DELETE /identity/v3/projects/8af7d2bbc6d94273bd39c81423d8cebe => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:30:59.787315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f25-8510-420e-8930-850b869c0d29 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.790343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f25-8510-420e-8930-850b869c0d29 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.790343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a5f25-8510-420e-8930-850b869c0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.809096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e35a5f25-8510-420e-8930-850b869c0d29 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:59.810777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e35a5f25-8510-420e-8930-850b869c0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:59.881676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1183/2348] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:30:59.890485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.890785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.891135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.907735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:59.913723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:59.914943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:59.917303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.917785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.918216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.927489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1166/2349] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:30:59.929651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:59.935065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:30:59.941144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b5460-709f-477c-9aec-83b0927fc245 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:59.941594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] MFA Rules not processed for user `578cd837d9fe442587f8c6c8528334c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:30:59.942981 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:30:59.946167 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:59.946776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.947059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.947741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.948361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:30:59.948699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:59.950944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:30:59.962522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1167/2350] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:30:59.967499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:30:59.978023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a85cb-c31a-4a46-8d87-5f224860dfe6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:30:59.984489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:30:59.984489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:30:59.984489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:30:59.984848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:30:59.985347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:30:59.987267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.002273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1184/2351] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:30:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:31:00.080110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1168/2352] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:30:59 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:31:00.084779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.097128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff9f2-25c1-4c96-9c1e-d054afc420eb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.154324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.154944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.155226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.155499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.156123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.158250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.257323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1185/2353] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:00 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:00.258476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.269736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40784c81-0758-45db-acf8-6ae8c875184c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.274395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.275121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.276000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.276918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.277579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.279163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.292458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1169/2354] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:00.293699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.304151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde1f6-3427-45da-b84b-b1e813c0d160 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.309725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.310371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.310651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.310922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.311593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.313250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.324966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1186/2355] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:00.326780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.328820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:00.328989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.339642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdccf5-ff1b-4b22-82c1-039110e1ed31 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.344279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.345034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.345314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.345592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] PATH_INFO: `/v3/projects/3bb1d8df8a82457ea1a5aa861593ce7d/users/ae642dfb02f647cf908475f9bbd32747/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.360874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb1d8df8a82457ea1a5aa861593ce7d, user_id=ae642dfb02f647cf908475f9bbd32747, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.363498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.387960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed483ca0-3334-4c38-98f7-315b5e7485a8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.391658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.392284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.392528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.392840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.394544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:00.405153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.406570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea0de8-290c-453a-81f3-aaa34c20345a service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.415047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1170/2356] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:00.454059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1187/2357] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:00 2022] PUT /identity/v3/projects/3bb1d8df8a82457ea1a5aa861593ce7d/users/ae642dfb02f647cf908475f9bbd32747/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:00.466151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.466438 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.466935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.495960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] MFA Rules not processed for user `ae642dfb02f647cf908475f9bbd32747`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:00.497287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:00.566210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1171/2358] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:00.570925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.638134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319df8-c218-4cb3-8385-9ea7a8c1ed5c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.643192 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.643805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.644085 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.644359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.645034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.647027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.662556 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:00.769032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1188/2359] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:00 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:00.773036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.783620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3c8f8-297f-4cd3-a690-b8a85ad76a62 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.827811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:00.839967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.840572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.840883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.841160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.841783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.843693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.897961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.899794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:00.956214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1172/2360] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:00 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:31:00.960014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.969972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56852e-2a04-49e0-ba89-916714d3b9f5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.975066 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.975734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.981672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.981672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.981876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01528a2-3758-44f8-9921-ed41943e3d87 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:00.981973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:00.981973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:00.983367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:00.983967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:00.984246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:00.984514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:00.986270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:00.989918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1173/2361] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:31:00.993968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:00.999410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:01.001041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:01.004821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ee45c-7b29-4f80-8961-9b19848f7d81 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:01.010333 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:01.010995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:01.011273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:01.011423 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1189/2362] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:31:01.013233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] PATH_INFO: `/v3/projects/df99551649e040db90b398934971157b/users/9f137e382fdc4d8ab552c7b4e36df676/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:01.028043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df99551649e040db90b398934971157b, user_id=9f137e382fdc4d8ab552c7b4e36df676, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:01.030804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:01.124080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1174/2363] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:00 2022] PUT /identity/v3/projects/df99551649e040db90b398934971157b/users/9f137e382fdc4d8ab552c7b4e36df676/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:01.128109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:01.147892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ddaae-0ba8-4da8-bae8-7206ada6f5bc service placement] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:01.203844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:01.204457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:01.204773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:01.205051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:01.205697 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:01.207331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de702f59-a39c-4d8a-ac5c-15570ab0b457 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:01.218214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1190/2364] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:01.222656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:01.233633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dcb1b-ca4c-4f1e-97f5-6cda2e7ae66e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:01.245478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:01.245478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:01.245478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:01.245478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] PATH_INFO: `/v3/projects/df99551649e040db90b398934971157b/users/9f137e382fdc4d8ab552c7b4e36df676/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:01.245873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df99551649e040db90b398934971157b, user_id=9f137e382fdc4d8ab552c7b4e36df676, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:01.245873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:01.255992 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1175/2365] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:01 2022] PUT /identity/v3/projects/df99551649e040db90b398934971157b/users/9f137e382fdc4d8ab552c7b4e36df676/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:01.266701 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:01.266990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:01.267306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:01.299203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] MFA Rules not processed for user `9f137e382fdc4d8ab552c7b4e36df676`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:01.299203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:01.368602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1191/2366] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:01.379811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:01.381595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:01.441901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e42c27-184c-46aa-abb9-2351e547c918 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:01.445880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:01.446531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:01.446812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:01.447088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:01.456164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:01.457628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723d9a0-f398-4741-b19d-1f6be192ac69 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:01.472736 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1176/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:02.629473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:02.633718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:02.691808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f58f0c-1e44-45dd-84cb-4e6ee367a447 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:02.696644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:02.697408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:02.697695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:02.697971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] PATH_INFO: `/v3/users/ae642dfb02f647cf908475f9bbd32747` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:02.699117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae642dfb02f647cf908475f9bbd32747)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:02.701093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:03.076773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae642dfb02f647cf908475f9bbd32747'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:03.076773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae642dfb02f647cf908475f9bbd32747'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:03.087261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae642dfb02f647cf908475f9bbd32747'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:03.089129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae642dfb02f647cf908475f9bbd32747'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:03.089765 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae642dfb02f647cf908475f9bbd32747'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:03.195957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1192/2368] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:02 2022] DELETE /identity/v3/users/ae642dfb02f647cf908475f9bbd32747 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:03.211182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:03.211696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:03.211866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:03.231655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:03.233020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:03.296040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1177/2369] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:03.305555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:03.307766 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:03.365997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd936dd4-a4cb-4202-a855-73f38025d9fd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:03.370385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:03.370987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:03.371269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:03.371548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:03.380562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:03.382013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:03.391622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1193/2370] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:03.848866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:03.850461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:03.866458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ec68a-bfa5-4f17-811f-8f181e23b5b8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:03.871418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:03.872059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:03.872341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:03.872617 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] PATH_INFO: `/v3/projects/3bb1d8df8a82457ea1a5aa861593ce7d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:03.873900 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb1d8df8a82457ea1a5aa861593ce7d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:03.876772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:04.003551 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:04.022809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb1d8df8a82457ea1a5aa861593ce7d', 'request_id': 'req-55923be2-3894-4a5d-a477-df9b46726cdd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.127215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb1d8df8a82457ea1a5aa861593ce7d', 'request_id': 'req-55923be2-3894-4a5d-a477-df9b46726cdd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.135387 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb1d8df8a82457ea1a5aa861593ce7d', 'request_id': 'req-55923be2-3894-4a5d-a477-df9b46726cdd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.146898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] The token cache is being invalidate because project 3bb1d8df8a82457ea1a5aa861593ce7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:04.147278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.148033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.149771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1178/2371] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:03 2022] DELETE /identity/v3/projects/3bb1d8df8a82457ea1a5aa861593ce7d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:04.154456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:04.164978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725c4ea-a751-4206-8beb-b43ab268a317 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:04.189396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:04.241625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:04.242286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:04.242695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:04.242974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] PATH_INFO: `/v3/users/9f137e382fdc4d8ab552c7b4e36df676` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:04.244388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f137e382fdc4d8ab552c7b4e36df676)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:04.249873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:04.500841 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f137e382fdc4d8ab552c7b4e36df676'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.507596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f137e382fdc4d8ab552c7b4e36df676'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.519490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f137e382fdc4d8ab552c7b4e36df676'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.520158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f137e382fdc4d8ab552c7b4e36df676'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.520787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6a900dad-91f3-497a-8fae-fe97b18ece86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f137e382fdc4d8ab552c7b4e36df676'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:04.680290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1194/2372] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:04 2022] DELETE /identity/v3/users/9f137e382fdc4d8ab552c7b4e36df676 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:05.186779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:05.188334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:05.207546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923be2-3894-4a5d-a477-df9b46726cdd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:05.274126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:05.274795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.275071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.275928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] PATH_INFO: `/v3/projects/df99551649e040db90b398934971157b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.277273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df99551649e040db90b398934971157b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:05.279183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:05.467225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df99551649e040db90b398934971157b', 'request_id': 'req-cbc232ca-d9d0-4df9-8e70-43424521cc0f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:05.544730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df99551649e040db90b398934971157b', 'request_id': 'req-cbc232ca-d9d0-4df9-8e70-43424521cc0f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:05.550855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df99551649e040db90b398934971157b', 'request_id': 'req-cbc232ca-d9d0-4df9-8e70-43424521cc0f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:05.562122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] The token cache is being invalidate because project df99551649e040db90b398934971157b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:05.562414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:05.563068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbc232ca-d9d0-4df9-8e70-43424521cc0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:05.564794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1179/2373] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:05 2022] DELETE /identity/v3/projects/df99551649e040db90b398934971157b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:05.569401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.570186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.570186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.592491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:05.594000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:05.656000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1195/2374] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:05.665340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.665659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.666042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.685750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:05.686958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:05.695048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1180/2375] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:05.700246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:05.709875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1547-f0cc-4ad9-a2a1-ec4141abb4f7 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:05.715539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:05.716129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.716404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.716688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.717367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:31:05.717669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:05.719236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:05.731027 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1196/2376] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:31:05.735214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:05.744635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880d288-8f96-454a-b4f1-935843770a80 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:05.749310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:05.749912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.750193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.750499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.751130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:05.753080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:05.853496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1181/2377] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:05 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:05.858164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:05.868434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7507-e28f-4827-a128-30c102dcf212 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:05.902293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:05.932116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:05.932772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:05.933113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:05.933387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:05.934016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:05.935917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.096939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1197/2378] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:05 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:06.098432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:06.111313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1b28c-9982-490d-9650-f89b5f9f64e2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:06.116256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:06.116978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.117325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.117689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.118516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:06.121543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.136259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1182/2379] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:06.140042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:06.164288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e663-dc01-4af9-86e7-ab4c1957cfa4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:06.170482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:06.171124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.171395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.171657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.172318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:06.178983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3104042-1424-446a-aea0-3693fc6060d6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.188012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1198/2380] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:31:06.189406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:06.196163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:06.200758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b673943-1964-4d2e-b2d9-4e0b64fa4ae8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:06.205756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:06.206436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.206717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.206990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] PATH_INFO: `/v3/projects/cdc2d17a0ce840698c058958bd3b3a8a/users/19e4acc3b06d4c409b4b5769dd5b2e72/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.223649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc2d17a0ce840698c058958bd3b3a8a, user_id=19e4acc3b06d4c409b4b5769dd5b2e72, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:06.226334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.389480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1183/2381] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:06 2022] PUT /identity/v3/projects/cdc2d17a0ce840698c058958bd3b3a8a/users/19e4acc3b06d4c409b4b5769dd5b2e72/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:06.401851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.402138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.402415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.432190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] MFA Rules not processed for user `19e4acc3b06d4c409b4b5769dd5b2e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:06.434165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:06.502966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1199/2382] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:06.514719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:06.516458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:06.585958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd843c7a-de4a-439c-953d-5e7adcfeef43 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:06.591606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:06.592258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.597152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.597152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.609514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:06.611246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26291592-fd4a-490c-9a16-b69c66c6ac5f service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.621056 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1184/2383] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:06.643071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:06.710737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237b8d-9cc5-4dfd-bf20-153237d49720 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:06.716255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:06.716931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:06.717219 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:06.717497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] PATH_INFO: `/v3/users/19e4acc3b06d4c409b4b5769dd5b2e72` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:06.718646 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e4acc3b06d4c409b4b5769dd5b2e72)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:06.720617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:06.992030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e4acc3b06d4c409b4b5769dd5b2e72'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:06.999032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e4acc3b06d4c409b4b5769dd5b2e72'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:07.010965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e4acc3b06d4c409b4b5769dd5b2e72'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:07.011847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e4acc3b06d4c409b4b5769dd5b2e72'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:07.013659 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:07.014371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e4acc3b06d4c409b4b5769dd5b2e72'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:07.122001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1200/2384] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:06 2022] DELETE /identity/v3/users/19e4acc3b06d4c409b4b5769dd5b2e72 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:07.130016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:07.130305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:07.130581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:07.149769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:07.151573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:07.210813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1185/2385] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:07.221628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:07.223165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:07.282333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37dd34-51f6-41ce-9437-0a3d57aad35f admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:07.285955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:07.286554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:07.286833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:07.287101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:07.296296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:07.297780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedf04d-05fc-49fa-9cbe-c0f7beec99df service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:07.308106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1201/2386] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:07.859237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:07.861076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:07.874913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea2400-d0b3-439d-95ef-c8169e3a79c0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:07.879645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:07.880277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:07.880782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:07.880960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] PATH_INFO: `/v3/projects/cdc2d17a0ce840698c058958bd3b3a8a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:07.882096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc2d17a0ce840698c058958bd3b3a8a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:07.884026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.054622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc2d17a0ce840698c058958bd3b3a8a', 'request_id': 'req-b6770e6f-16e9-4280-a489-72aed7ec41e8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:08.235701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc2d17a0ce840698c058958bd3b3a8a', 'request_id': 'req-b6770e6f-16e9-4280-a489-72aed7ec41e8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:08.241829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc2d17a0ce840698c058958bd3b3a8a', 'request_id': 'req-b6770e6f-16e9-4280-a489-72aed7ec41e8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:08.253729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] The token cache is being invalidate because project cdc2d17a0ce840698c058958bd3b3a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:08.254018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:08.254688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-b6770e6f-16e9-4280-a489-72aed7ec41e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:08.256224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1186/2387] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:07 2022] DELETE /identity/v3/projects/cdc2d17a0ce840698c058958bd3b3a8a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:08.261124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.261596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.261714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.281975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:08.283201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:08.343082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1202/2388] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:08.350730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.351013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.351287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.366269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:08.371702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:08.372933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:08.380745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1187/2389] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:08.392272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.402878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198253c4-f84d-4cac-a8e0-aca6ca8b35fe None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.407504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.408101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.408384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.408747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.409339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:31:08.409644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.411268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.423044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1203/2390] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:31:08.427308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.436606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba2313-78ab-42eb-b264-a5ea1be3005a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.441220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.441833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.442108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.442376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.443004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.445182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.597720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1188/2391] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:08 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:31:08.601760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.611930 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cabb9-ed90-428b-ba53-b099ae2dcc7e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.657323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:08.668890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.669491 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.669767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.670035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.670652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.672548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.814000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1204/2392] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:08 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:08.818314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.830043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f983c20-7832-4781-8243-4821a28d37b9 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.834554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.835147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.835427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.835698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.836337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.838258 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.848270 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1189/2393] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:08.852149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.862940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7008f5-a4bd-4b9e-8b3b-2e789b3f17f2 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.867668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.868462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.868541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.868832 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.870107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.871726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73120275-69b3-4279-9e0d-d57a15007ab7 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:08.882500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1205/2394] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:08.887954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:08.897953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd673b8f-7773-430b-a2e1-dac7575310a2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:08.902595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:08.903238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:08.903515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:08.906016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] PATH_INFO: `/v3/projects/ecd3e1e5b7044f3aa46eeb893a3a3e04/users/0502cde38d394d2eabf87ce25dfc2e84/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:08.918399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd3e1e5b7044f3aa46eeb893a3a3e04, user_id=0502cde38d394d2eabf87ce25dfc2e84, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:08.921103 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:09.028210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1190/2395] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:08 2022] PUT /identity/v3/projects/ecd3e1e5b7044f3aa46eeb893a3a3e04/users/0502cde38d394d2eabf87ce25dfc2e84/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:09.043669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:09.043954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:09.044228 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:09.074481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] MFA Rules not processed for user `0502cde38d394d2eabf87ce25dfc2e84`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:09.075699 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:09.139526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1206/2396] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:09.150228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:09.152472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:09.209919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8d842-28e4-4d08-892e-abc35e38be47 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:09.214216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:09.214859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:09.215140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:09.215413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:09.223957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:09.225452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:09.234838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1191/2397] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:09.435570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:09.437050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:09.512757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26119dd-3d03-4154-a036-673f2a6c4e76 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:09.518451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:09.519066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:09.519348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:09.519661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:09.521436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:09.578523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:09.582331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:09.583912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:09.595563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1207/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:12.324610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:12.324610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:12.333488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafbf4d-c85c-4e6a-ade2-0dbf58975770 service glance] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:12.341088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:12.341749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:12.342034 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:12.342311 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:12.344109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:12.361047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:12.362819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:12.375170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1192/2399] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:17.622960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:17.634311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57b9c-e943-4348-9f47-ee813c407e34 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:17.641008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:17.641622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:17.641905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:17.642183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:17.643953 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:17.659534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:17.661261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:17.672682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1208/2400] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:22.370162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:23.541945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:23.543404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:23.616024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9277c11-db02-4d85-a31a-e28695274b88 service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:23.621569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:23.622182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:23.622461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:23.622735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:23.624464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:23.635602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:23.637865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:23.647547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1193/2401] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:23.665999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:23.667438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:23.738873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75555daa-24e6-4bb7-ba86-d72d535ce597 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:23.743532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:23.744159 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:23.744440 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:23.744739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] PATH_INFO: `/v3/users/5e3c891f2f3148f4aaba41a14d507cdf` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:23.745884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3c891f2f3148f4aaba41a14d507cdf)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:23.748522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:23.957051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3c891f2f3148f4aaba41a14d507cdf'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:23.966604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3c891f2f3148f4aaba41a14d507cdf'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:23.980540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3c891f2f3148f4aaba41a14d507cdf'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:23.981314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3c891f2f3148f4aaba41a14d507cdf'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:23.982098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3c891f2f3148f4aaba41a14d507cdf'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.082151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1209/2402] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:23 2022] DELETE /identity/v3/users/5e3c891f2f3148f4aaba41a14d507cdf => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:24.375534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:24.391704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb33027-2836-4668-9c21-9f6255f6bc4f service placement] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:24.452288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:24.452956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:24.453232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:24.453505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] PATH_INFO: `/v3/projects/7203c52ae1f946bd9717a554316e4f4c` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:24.454663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7203c52ae1f946bd9717a554316e4f4c)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:24.456763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:24.596568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7203c52ae1f946bd9717a554316e4f4c', 'request_id': 'req-01d130c3-b82f-4aaa-b4e6-6afe30375f13'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.673767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7203c52ae1f946bd9717a554316e4f4c', 'request_id': 'req-01d130c3-b82f-4aaa-b4e6-6afe30375f13'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.680570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7203c52ae1f946bd9717a554316e4f4c', 'request_id': 'req-01d130c3-b82f-4aaa-b4e6-6afe30375f13'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.695317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] The token cache is being invalidate because project 7203c52ae1f946bd9717a554316e4f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:24.695607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.696266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-01d130c3-b82f-4aaa-b4e6-6afe30375f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:24.698164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1194/2403] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:24 2022] DELETE /identity/v3/projects/7203c52ae1f946bd9717a554316e4f4c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:28.853555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:28.854951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:28.868274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41b8b2-91af-4bae-9edc-a537c54b2d70 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:28.924570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:28.925215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:28.925494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:28.925767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] PATH_INFO: `/v3/users/0502cde38d394d2eabf87ce25dfc2e84` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:28.926879 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0502cde38d394d2eabf87ce25dfc2e84)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:28.928827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:28.930251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:29.146631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0502cde38d394d2eabf87ce25dfc2e84'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:29.153626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0502cde38d394d2eabf87ce25dfc2e84'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:29.165231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0502cde38d394d2eabf87ce25dfc2e84'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:29.166067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0502cde38d394d2eabf87ce25dfc2e84'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:29.167301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0502cde38d394d2eabf87ce25dfc2e84'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:29.253274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1210/2404] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:28 2022] DELETE /identity/v3/users/0502cde38d394d2eabf87ce25dfc2e84 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:29.270413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:29.270903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:29.271058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:29.290854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:29.292044 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:29.306874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:29.355556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1195/2405] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:29.366320 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:29.367816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:29.426011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf9b3-724a-4da3-99da-678907ae6cf6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:29.429618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:29.430245 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:29.430527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:29.430800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:29.439989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:29.441442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889a97b-bbc9-46ca-83aa-436a528d71c9 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:29.452133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1211/2406] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:29.884075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:29.887762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:29.899847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce148e63-c1fc-426a-9a65-6294852fe4a2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:29.905306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:29.905948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:29.906230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:29.906506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] PATH_INFO: `/v3/projects/ecd3e1e5b7044f3aa46eeb893a3a3e04` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:29.907653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd3e1e5b7044f3aa46eeb893a3a3e04)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:29.909675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.053248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd3e1e5b7044f3aa46eeb893a3a3e04', 'request_id': 'req-65939aa3-1ee5-40d5-81dd-8f662aff8722'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:30.144283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd3e1e5b7044f3aa46eeb893a3a3e04', 'request_id': 'req-65939aa3-1ee5-40d5-81dd-8f662aff8722'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:30.150112 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd3e1e5b7044f3aa46eeb893a3a3e04', 'request_id': 'req-65939aa3-1ee5-40d5-81dd-8f662aff8722'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:30.161092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] The token cache is being invalidate because project ecd3e1e5b7044f3aa46eeb893a3a3e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:30.161383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:30.162005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-65939aa3-1ee5-40d5-81dd-8f662aff8722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:30.163514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1196/2407] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:29 2022] DELETE /identity/v3/projects/ecd3e1e5b7044f3aa46eeb893a3a3e04 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:30.169201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.169715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.169715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.195517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:30.196739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:30.257008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1212/2408] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:30.264611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.264925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.265230 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.284755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:30.285978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:30.294704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1197/2409] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:30.299061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.309350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2cc96-568f-4aa0-9b01-01866c7cc87f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.313909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.314502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.314782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.315052 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.315692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:31:30.315996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.317623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.330230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1213/2410] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:31:30.334525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.344853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cc15d-9dd1-4ad7-bb40-11efb69ea041 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.349484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.350086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.350367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.350652 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.351269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.353205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.443110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1198/2411] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:30.447518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.458511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41e01-80d4-4d77-84c4-7c9295d6d42e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.463271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:30.515656 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.516194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.516473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.516773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.517405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.519419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.640450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1214/2412] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:30.644649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.654896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58daf-d84f-411f-9012-1f502031f75e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.659349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.659933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.660208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.660479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.661162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.662739 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.672511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1199/2413] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:30.676357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.687460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410828e-be1e-480d-9f9b-65462d17178d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.692181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.692803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.693085 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.693358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.693995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.695830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c14087-b625-4146-bd28-5f009ff9680f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.707769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1215/2414] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:30.711658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.721487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1cc8-c11c-45bd-99b1-f572feb444d1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:30.725949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:30.726582 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.726856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.727123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] PATH_INFO: `/v3/projects/ff6213349ee8431382a7444d359e0c2c/users/55e3f90bf54f419eb2efa916c01953b8/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.740932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6213349ee8431382a7444d359e0c2c, user_id=55e3f90bf54f419eb2efa916c01953b8, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:30.743586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:30.842362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1200/2415] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:30 2022] PUT /identity/v3/projects/ff6213349ee8431382a7444d359e0c2c/users/55e3f90bf54f419eb2efa916c01953b8/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:30.851907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:30.854095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:30.854095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:30.881253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] MFA Rules not processed for user `55e3f90bf54f419eb2efa916c01953b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:30.882437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:30.944347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1216/2416] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:30.949176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:30.991138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:31.010639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea97dd-c165-4328-a7b8-c866c7dbe502 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.015576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.016183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.016467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.016788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.017461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.019910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.142788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1201/2417] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:30 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:31.144110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.155533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc84c4b-99f2-435b-bc9c-0ffa88700cec None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.213539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.214181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.214455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.214726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.216374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.218337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.299294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1217/2418] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:31 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:31.303343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.313718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0f912-7a17-4db0-a8f2-90d24c557e46 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.324244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.325144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.325429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.325727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.326384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.328040 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.338553 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1202/2419] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:31.343445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.353610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c694-3cc4-451f-84df-248ebc7719c4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.358882 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.359548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.359833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.360112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] PATH_INFO: `/v3/projects/8634124e87134713bf6d84bd285d6b28/users/b45a5cbcf4a045a292c4d198c5c6337b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.362382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:31.375385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8634124e87134713bf6d84bd285d6b28, user_id=b45a5cbcf4a045a292c4d198c5c6337b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.378182 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.453176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1218/2420] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:31 2022] PUT /identity/v3/projects/8634124e87134713bf6d84bd285d6b28/users/b45a5cbcf4a045a292c4d198c5c6337b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:31.457278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.468036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717e58f-18ef-40c0-8c4c-2ae43cc30c3f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.523957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.524561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.524868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.525144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.525784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.527892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d9733-c0ac-4c7c-a1f4-0c571faef841 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.537683 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1203/2421] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:31.541818 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.552966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83df147-e930-40d4-b140-dbde8bbc2948 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.557517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.558156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.558430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.558753 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] PATH_INFO: `/v3/projects/8634124e87134713bf6d84bd285d6b28/users/b45a5cbcf4a045a292c4d198c5c6337b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.560741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8634124e87134713bf6d84bd285d6b28, user_id=b45a5cbcf4a045a292c4d198c5c6337b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.563388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.573693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1219/2422] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:31 2022] PUT /identity/v3/projects/8634124e87134713bf6d84bd285d6b28/users/b45a5cbcf4a045a292c4d198c5c6337b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:31.585384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.585677 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.585956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.615207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] MFA Rules not processed for user `b45a5cbcf4a045a292c4d198c5c6337b`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:31.616425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:31.682868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1204/2423] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:31.694165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:31.701010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:31.753691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05624cee-7e78-4d9b-9996-1e2d4509dc3e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:31.757402 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:31.757996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:31.758296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:31.758566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:31.815428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:31.816922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9344c-302d-45f2-805d-a682f3cf47e0 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:31.826530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1220/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:34.954577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:34.993871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:35.016863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedd675-4fb3-4cc5-b20d-71708a4935e4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:35.021385 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:35.022011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:35.022293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:35.022566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] PATH_INFO: `/v3/users/55e3f90bf54f419eb2efa916c01953b8` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:35.023730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e3f90bf54f419eb2efa916c01953b8)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:35.025710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:35.246624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e3f90bf54f419eb2efa916c01953b8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:35.253755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e3f90bf54f419eb2efa916c01953b8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:35.266241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e3f90bf54f419eb2efa916c01953b8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:35.266926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e3f90bf54f419eb2efa916c01953b8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:35.267521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e3f90bf54f419eb2efa916c01953b8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:35.359346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1205/2425] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:34 2022] DELETE /identity/v3/users/55e3f90bf54f419eb2efa916c01953b8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:35.370498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:35.371011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:35.371011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:35.391011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:35.392218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:35.452627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1221/2426] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:35.462822 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:35.464307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:35.524210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8514f-af07-494c-91ca-7d950323defe admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:35.527955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:35.528561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:35.528871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:35.529145 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:35.538318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:35.539730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:35.549883 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1206/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:36.082566 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:36.083989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:36.097639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29760c-efc4-4f06-aeda-af2b12db3a73 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:36.102914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:36.103562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:36.103839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:36.104114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] PATH_INFO: `/v3/projects/ff6213349ee8431382a7444d359e0c2c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:36.105331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6213349ee8431382a7444d359e0c2c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:36.108037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:36.132174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:36.255432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6213349ee8431382a7444d359e0c2c', 'request_id': 'req-ddfb78d5-2e6f-4493-abaa-a0150b583635'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.350645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6213349ee8431382a7444d359e0c2c', 'request_id': 'req-ddfb78d5-2e6f-4493-abaa-a0150b583635'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.358629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6213349ee8431382a7444d359e0c2c', 'request_id': 'req-ddfb78d5-2e6f-4493-abaa-a0150b583635'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.370073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] The token cache is being invalidate because project ff6213349ee8431382a7444d359e0c2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:36.370530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.372337 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ddfb78d5-2e6f-4493-abaa-a0150b583635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.373855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1222/2428] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:36 2022] DELETE /identity/v3/projects/ff6213349ee8431382a7444d359e0c2c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:36.378130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:36.388484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f111c-c072-454a-910f-6ca67d105909 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:36.446825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:36.453548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:36.454216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:36.454499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:36.454777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] PATH_INFO: `/v3/users/b45a5cbcf4a045a292c4d198c5c6337b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:36.455968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45a5cbcf4a045a292c4d198c5c6337b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:36.457986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:36.724476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45a5cbcf4a045a292c4d198c5c6337b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.732004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45a5cbcf4a045a292c4d198c5c6337b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.746898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45a5cbcf4a045a292c4d198c5c6337b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.748160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45a5cbcf4a045a292c4d198c5c6337b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.748873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45a5cbcf4a045a292c4d198c5c6337b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:36.803853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:36.804136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:36.804410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:36.825377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:36.826609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:36.846497 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1207/2429] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:36 2022] DELETE /identity/v3/users/b45a5cbcf4a045a292c4d198c5c6337b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:36.892682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1223/2430] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:31:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:31:37.039445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:37.040995 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:37.113924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ace252-e7f2-44c3-8ae0-11cdc3057e8e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:37.119795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:37.120535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:37.120854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:37.121228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:37.192880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:37.194521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535f71a-1fff-48be-9d3e-179137be75ca service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:37.203345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1208/2431] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:31:37.532194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:37.532194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:37.550098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25dcd-6a55-44da-b7b2-eb78dca81b4c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:37.626703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:37.627601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:37.628267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:37.628653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] PATH_INFO: `/v3/projects/8634124e87134713bf6d84bd285d6b28` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:37.630079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8634124e87134713bf6d84bd285d6b28)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:37.632606 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:37.813021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8634124e87134713bf6d84bd285d6b28', 'request_id': 'req-00091fd5-9cb4-4411-ad0f-e600b6119a99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:37.819209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:37.891158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8634124e87134713bf6d84bd285d6b28', 'request_id': 'req-00091fd5-9cb4-4411-ad0f-e600b6119a99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:37.901807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8634124e87134713bf6d84bd285d6b28', 'request_id': 'req-00091fd5-9cb4-4411-ad0f-e600b6119a99'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:37.918072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] The token cache is being invalidate because project 8634124e87134713bf6d84bd285d6b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:37.918408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:37.919470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-00091fd5-9cb4-4411-ad0f-e600b6119a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:37.924756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1224/2432] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:37 2022] DELETE /identity/v3/projects/8634124e87134713bf6d84bd285d6b28 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:37.925989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:37.926471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:37.926615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:37.947267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:37.948547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:37.962731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:38.019532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1209/2433] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:38.028299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.028598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.029000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.052246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:38.053743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:38.063209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1225/2434] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:38.067717 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.079546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8ef5f-43b4-4200-a99b-225c3779d1c2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.084852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.085531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.086179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.090234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.091123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:31:38.091538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.093447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.109291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1210/2435] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:31:38.111160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.122612 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9dd50-9036-456d-b571-4bbfddd1e90a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.127883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.128631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.128947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.131800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.132560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.134674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.239913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1226/2436] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:38 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:38.244299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.265453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690705e-caab-42f6-a5f5-321d5d574cfe admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.383210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.383961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.384246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.384525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.385812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.390129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.508797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1211/2437] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:38 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:38.513610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.524660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a47fc-12d3-4dff-b9e4-9c8bef207df2 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.530631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.531257 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.531589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.531867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.532521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.534177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.549537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1227/2438] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:38.550656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.562979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169d049-d525-4ae3-853d-e00e860fabe5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.568518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.572443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.574137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.574137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.575512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.577917 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d828393-264e-44b3-8ca9-1fb587fa0422 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.600936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1212/2439] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:38.604361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.619960 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124119-8709-453d-9fba-ceac81e036eb admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.625977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.626639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.627015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.627482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] PATH_INFO: `/v3/projects/deb60c07c40d4340b335b52807253d79/users/ea18b39f54eb42f2a75933a6fb5f8f2f/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.648080 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb60c07c40d4340b335b52807253d79, user_id=ea18b39f54eb42f2a75933a6fb5f8f2f, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.650815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:38.722798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1228/2440] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:38 2022] PUT /identity/v3/projects/deb60c07c40d4340b335b52807253d79/users/ea18b39f54eb42f2a75933a6fb5f8f2f/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:38.734349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.734660 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.734906 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.768392 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] MFA Rules not processed for user `ea18b39f54eb42f2a75933a6fb5f8f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:38.769723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:38.848511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1213/2441] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:38.853039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:38.936557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da745b67-3265-40ad-8b2d-84b85e63e5dc admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:38.941833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:38.942682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:38.943185 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:38.943662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:38.944286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:38.946528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.038695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1229/2442] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:38 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:39.046262 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.064213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb02be8-a135-4552-ab13-df55aadcf924 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.074977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:39.141008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.141636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.141915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.142191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.142832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.144804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.246747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1214/2443] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:39 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:39.253134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.274947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94abf76-aa87-4b9a-bd3b-4264351629f5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.287114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.287769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.288057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.288336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.289044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.290669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.307177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1230/2444] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:39.311316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.323910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab266-35b1-4ee2-b188-2eb8ca12fc72 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.334285 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.334894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.335184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.335735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.336710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.338398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf584a78-e8b2-4b33-80e9-f46e0652b0cf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.361823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1215/2445] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:39.365788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.377278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216f67-6075-44da-8853-3f1e0670c199 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.382254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.382927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.383215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.383481 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] PATH_INFO: `/v3/projects/3736202954eb45bcae664326f7eb862d/users/24c16c17c58c4489b92e6006d4944299/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.401106 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3736202954eb45bcae664326f7eb862d, user_id=24c16c17c58c4489b92e6006d4944299, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.404616 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.516008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1231/2446] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:39 2022] PUT /identity/v3/projects/3736202954eb45bcae664326f7eb862d/users/24c16c17c58c4489b92e6006d4944299/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:39.534125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.534125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.534125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.565193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] MFA Rules not processed for user `24c16c17c58c4489b92e6006d4944299`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:39.566794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:39.663789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1216/2447] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:39.669005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.707267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:39.756899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941c7cf-4f9d-4428-9e84-17b12415af19 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.762969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.763596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.763931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.764207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.765057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.767230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:39.888722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1232/2448] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:39 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:39.892252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:39.908342 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1c1cf-1c40-4f25-9627-c7425bc83beb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:39.977495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:39.978377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:39.978727 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:39.979139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:39.979869 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:39.981993 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:40.063843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1217/2449] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:39 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:40.068066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:40.080761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e958ba-f32e-44e0-a055-a0d0ac61cd84 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:40.085457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:40.086066 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.086343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.086615 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:40.087250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:40.088888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:40.104758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1233/2450] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:40.106153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:40.116587 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d7e8-32ba-4eae-9b8e-860fc160984c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:40.121280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:40.121935 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.122305 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.122583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] PATH_INFO: `/v3/projects/e4b02a39494c4b55994fbd14868403ec/users/4d886efe3f924a9983ed179f998a0cbe/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:40.131848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:40.140594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b02a39494c4b55994fbd14868403ec, user_id=4d886efe3f924a9983ed179f998a0cbe, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:40.144521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:40.487840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1218/2451] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:40 2022] PUT /identity/v3/projects/e4b02a39494c4b55994fbd14868403ec/users/4d886efe3f924a9983ed179f998a0cbe/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:40.491996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:40.502764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade60f3-74e5-48bf-9ccc-0b23e0882198 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:40.569424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:40.570047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.570580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.570859 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:40.571579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:40.573246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc746d7-63f7-49fb-8969-fb4b87e6407b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:40.592964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1234/2452] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:40.600046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:40.610414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdbffa-1ca8-4193-8d01-87b2758b4041 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:40.615890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:40.616569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.616963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.617277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] PATH_INFO: `/v3/projects/e4b02a39494c4b55994fbd14868403ec/users/4d886efe3f924a9983ed179f998a0cbe/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:40.619293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b02a39494c4b55994fbd14868403ec, user_id=4d886efe3f924a9983ed179f998a0cbe, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:40.622704 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:40.635891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1219/2453] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:40 2022] PUT /identity/v3/projects/e4b02a39494c4b55994fbd14868403ec/users/4d886efe3f924a9983ed179f998a0cbe/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:40.644622 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.644974 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.645255 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:40.681963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] MFA Rules not processed for user `4d886efe3f924a9983ed179f998a0cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:40.684251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:40.819123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1235/2454] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:40.830737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:40.832484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:40.932976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4d540-28e4-4af9-8582-0869aa68fac5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:40.937830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:40.938464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:40.938746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:40.939027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:41.008375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:41.009777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:41.024825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1220/2455] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:43.130568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:43.132033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:43.147702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a764b7f-e346-49a1-ae76-eef8d528d199 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:43.151360 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:43.151971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:43.152301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:43.152575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:43.174448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:43.210629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:43.429861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:43.429861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1236/2456] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:45.426413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:45.435785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a3923-f331-4291-b253-67580fd05db9 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:45.439687 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:45.440363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:45.440653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:45.440949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:45.452020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:45.453460 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fb2c-0b8a-41b9-9f32-ec18488db7f5 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:45.468811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1221/2457] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:50.501214 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:50.565978 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ec5ed-0df0-428c-92a1-dcce30ce866a service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:50.570900 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:50.571651 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:50.571940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:50.572331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] PATH_INFO: `/v3/users/ea18b39f54eb42f2a75933a6fb5f8f2f` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:50.573937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea18b39f54eb42f2a75933a6fb5f8f2f)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:50.575411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:50.744386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea18b39f54eb42f2a75933a6fb5f8f2f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.752954 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea18b39f54eb42f2a75933a6fb5f8f2f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.767604 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea18b39f54eb42f2a75933a6fb5f8f2f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.768330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea18b39f54eb42f2a75933a6fb5f8f2f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.768972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea18b39f54eb42f2a75933a6fb5f8f2f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:50.864427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1237/2458] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:50 2022] DELETE /identity/v3/users/ea18b39f54eb42f2a75933a6fb5f8f2f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:50.876626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:50.876626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:50.876626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:50.888484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:50.889740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:50.951922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1222/2459] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:31:50.961479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:50.962912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:51.010571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:51.025589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790cb6-a773-4261-812f-da471c5330ab admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:51.026062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:51.026671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:51.026947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:51.027243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:51.036372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:51.037837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:51.048716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1238/2460] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:51.531064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:51.533071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:51.548314 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4dd061-694c-48c1-a390-9cfd427b2200 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:51.551865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:51.552518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:51.552845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:51.553122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] PATH_INFO: `/v3/projects/deb60c07c40d4340b335b52807253d79` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:51.554978 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb60c07c40d4340b335b52807253d79)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:51.556985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:51.569015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:51.773527 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb60c07c40d4340b335b52807253d79', 'request_id': 'req-999c1a2a-4844-44bc-8a52-a17ca55b698c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:51.881484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb60c07c40d4340b335b52807253d79', 'request_id': 'req-999c1a2a-4844-44bc-8a52-a17ca55b698c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:51.887847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb60c07c40d4340b335b52807253d79', 'request_id': 'req-999c1a2a-4844-44bc-8a52-a17ca55b698c'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:51.898896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] The token cache is being invalidate because project deb60c07c40d4340b335b52807253d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:51.899185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:51.899808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:51.901326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1223/2461] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:51 2022] DELETE /identity/v3/projects/deb60c07c40d4340b335b52807253d79 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:51.905500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:51.915398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1463fd5-6080-4bcf-b79a-3cd572ddb273 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:51.971846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:51.972472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:51.972783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:51.973054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] PATH_INFO: `/v3/users/24c16c17c58c4489b92e6006d4944299` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:51.974158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c16c17c58c4489b92e6006d4944299)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:51.976065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:52.150059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c16c17c58c4489b92e6006d4944299'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:52.156940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c16c17c58c4489b92e6006d4944299'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:52.169230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c16c17c58c4489b92e6006d4944299'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:52.169907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c16c17c58c4489b92e6006d4944299'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:52.170638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c16c17c58c4489b92e6006d4944299'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:52.246593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1239/2462] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:51 2022] DELETE /identity/v3/users/24c16c17c58c4489b92e6006d4944299 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:52.920120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:52.921756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:52.936654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c1a2a-4844-44bc-8a52-a17ca55b698c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:53.003817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:53.004494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:53.004805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:53.005079 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] PATH_INFO: `/v3/projects/3736202954eb45bcae664326f7eb862d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:53.006207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3736202954eb45bcae664326f7eb862d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:53.008324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:53.141248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3736202954eb45bcae664326f7eb862d', 'request_id': 'req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.241666 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3736202954eb45bcae664326f7eb862d', 'request_id': 'req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.247878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3736202954eb45bcae664326f7eb862d', 'request_id': 'req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.259091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] The token cache is being invalidate because project 3736202954eb45bcae664326f7eb862d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:53.259382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.260003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.261519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1224/2463] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:52 2022] DELETE /identity/v3/projects/3736202954eb45bcae664326f7eb862d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:53.265694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:53.267150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:53.285166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb88e46-e6be-4bbf-819c-76ec4b933d0b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:53.345113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:53.345764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:53.346036 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:53.346330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] PATH_INFO: `/v3/users/4d886efe3f924a9983ed179f998a0cbe` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:53.347457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d886efe3f924a9983ed179f998a0cbe)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:53.349449 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:53.354209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:53.555964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d886efe3f924a9983ed179f998a0cbe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.563658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d886efe3f924a9983ed179f998a0cbe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.577727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d886efe3f924a9983ed179f998a0cbe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.578453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d886efe3f924a9983ed179f998a0cbe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.579117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e139220e-1dcc-42f8-a7d2-ff2c9909b804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d886efe3f924a9983ed179f998a0cbe'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:53.674846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1240/2464] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:53 2022] DELETE /identity/v3/users/4d886efe3f924a9983ed179f998a0cbe => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:54.271881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:54.273348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:54.286123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8b1fd-bdc3-4683-a3be-b416eb95f07d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:54.321465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:54.355152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:54.355892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:54.356169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:54.356444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] PATH_INFO: `/v3/projects/e4b02a39494c4b55994fbd14868403ec` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:54.357632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b02a39494c4b55994fbd14868403ec)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:54.359601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:54.528090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b02a39494c4b55994fbd14868403ec', 'request_id': 'req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:54.625199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b02a39494c4b55994fbd14868403ec', 'request_id': 'req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:54.631776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b02a39494c4b55994fbd14868403ec', 'request_id': 'req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:54.644137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] The token cache is being invalidate because project e4b02a39494c4b55994fbd14868403ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:31:54.644568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:54.645435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8ba4a9ad-39ef-4784-bae9-d88e602fba9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:54.652023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1225/2465] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:31:54 2022] DELETE /identity/v3/projects/e4b02a39494c4b55994fbd14868403ec => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:54.654536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:54.655013 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:54.655099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:54.677948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:54.679223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:54.773873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1241/2466] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:54.783857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:54.784442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:54.785138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:54.816162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:54.816162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:54.836756 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1226/2467] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:54.897043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:54.907436 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5938f-2727-45ff-92da-20368b7c4872 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:54.912695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:54.913316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:54.913599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:54.913875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:54.914576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:31:54.914886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:54.916631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:54.936463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1242/2468] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:31:54.936566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:54.945130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed232256-2733-4c9c-b0cc-ab075e932128 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:54.950475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:54.951165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:54.951448 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:54.951724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:54.952352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:54.954528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.088459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1227/2469] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:31:54 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:55.093321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:55.103153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cafb19-996b-48fb-8849-40419f36241e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:55.173035 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:55.173950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.174594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.175148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.175897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:55.178469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.281328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:55.286713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1243/2470] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:31:55 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:31:55.291772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:55.303655 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3356499-7ae6-4817-8968-c3582436203c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:55.318113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:55.319328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.320100 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.320442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.321595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:55.323542 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.334633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1228/2471] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:55.338718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:55.349628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a8274-15f7-448b-86ea-40321a8aad8c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:55.354498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:55.355097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.358582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.358582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.358582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:55.359050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2766b-7951-4379-816e-5f09f31e46d4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.372129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1244/2472] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:31:55.376169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:55.387432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc71d6-9bc3-4f29-84a1-3596ebe19ff1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:55.392235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:55.392927 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.393216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.393491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] PATH_INFO: `/v3/projects/fc839cb32ac3477bad891ae495265dd1/users/381a92a523d748628674293b40061b3d/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.408941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc839cb32ac3477bad891ae495265dd1, user_id=381a92a523d748628674293b40061b3d, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:55.412264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.505623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1229/2473] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:31:55 2022] PUT /identity/v3/projects/fc839cb32ac3477bad891ae495265dd1/users/381a92a523d748628674293b40061b3d/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:31:55.516240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.516526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.516839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.563089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] MFA Rules not processed for user `381a92a523d748628674293b40061b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:55.564368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:55.670927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1245/2474] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:31:55.684190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:55.687078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:31:55.755071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27449df3-8ee4-4054-9b6a-c05f7cd9d967 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:55.760250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:55.760972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:55.761254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:55.762132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:55.771549 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:55.773090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2dbd8b-53f0-490c-98cc-e1ad4fe50fa9 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:55.777903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:55.785612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1230/2475] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:31:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:31:59.473119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:31:59.539171 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a70371-24b4-4d14-9f17-2ec6190113d3 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:31:59.544595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:31:59.545277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:59.545557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:59.545827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] PATH_INFO: `/v3/users/381a92a523d748628674293b40061b3d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:59.546969 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=381a92a523d748628674293b40061b3d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:31:59.548938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:31:59.781737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381a92a523d748628674293b40061b3d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:59.788778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381a92a523d748628674293b40061b3d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:59.802030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381a92a523d748628674293b40061b3d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:59.802779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381a92a523d748628674293b40061b3d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:59.803407 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381a92a523d748628674293b40061b3d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:31:59.905371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:31:59.910800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1246/2476] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:31:59 2022] DELETE /identity/v3/users/381a92a523d748628674293b40061b3d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:31:59.920181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d68a-865c-4b09-bef1-6b5668969129 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:31:59.920722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d68a-865c-4b09-bef1-6b5668969129 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:31:59.920826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d68a-865c-4b09-bef1-6b5668969129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:31:59.940941 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d312d68a-865c-4b09-bef1-6b5668969129 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:31:59.942179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d312d68a-865c-4b09-bef1-6b5668969129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:00.005964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1231/2477] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:00.016617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:00.018783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:00.079367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43ec9b-6eef-489e-9d0a-55b5108fae1f admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:00.083065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:00.083677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:00.084070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:00.084349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:00.093674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:00.095133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:00.104774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1247/2478] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:00.626781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:00.627067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:00.627346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:00.648001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:00.649251 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:00.816364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:00.818112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:00.835269 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1232/2479] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:32:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:00.836316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133c91-f64f-4924-bf70-458a5e0e4fae service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:00.842445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:00.844470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:00.844470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:00.844470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] PATH_INFO: `/v3/projects/fc839cb32ac3477bad891ae495265dd1` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:00.850097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc839cb32ac3477bad891ae495265dd1)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:00.850555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:00.852201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:00.852330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:00.928612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01aa04-3462-4a50-b54d-f72cbda1fcc1 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:00.934064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:00.934766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:00.935098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:00.935373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:00.943889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:00.945483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb12d-75b4-424b-907d-2f913ce1bf49 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:00.954568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1233/2480] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:32:00.995395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc839cb32ac3477bad891ae495265dd1', 'request_id': 'req-aacba465-f1ff-4f1a-8348-a1c4339fe250'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:01.122708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc839cb32ac3477bad891ae495265dd1', 'request_id': 'req-aacba465-f1ff-4f1a-8348-a1c4339fe250'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:01.133178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc839cb32ac3477bad891ae495265dd1', 'request_id': 'req-aacba465-f1ff-4f1a-8348-a1c4339fe250'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:01.173386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] The token cache is being invalidate because project fc839cb32ac3477bad891ae495265dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:01.174670 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:01.176705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aacba465-f1ff-4f1a-8348-a1c4339fe250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:01.179720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1248/2481] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:00 2022] DELETE /identity/v3/projects/fc839cb32ac3477bad891ae495265dd1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:01.187613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.195163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.195163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.223373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:01.224982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:01.284254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:01.335977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1234/2482] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:01.345181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.345476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.345951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.370031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:01.372568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:01.382314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1249/2483] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:01.387277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.405528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6b2a-8c45-4d77-9e36-02d6b76d6009 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.410630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.411324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.411796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.412179 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.413413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:01.413593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.416272 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:01.436608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1235/2484] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:01.441471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.454302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76440bca-1d0b-4fea-b0dc-e6999cd6f479 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.466915 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.468424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.468828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.469126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.469891 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.472190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:01.627903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1250/2485] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:01 2022] POST /identity/v3/projects => generated 399 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:01.629227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.642499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2fe5f-384d-426d-923f-fa529904beaf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.733095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.734000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.734291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.734849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.735635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.737979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:01.836538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1236/2486] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:01 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:01.841071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.852810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eda2e-5c18-4297-a6e0-7df8b0d653be admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.858682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.859193 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.859536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.859712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.860361 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.862031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:01.878138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1251/2487] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:01.879688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.891634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623431e-d704-40fd-8713-aa9de54f8233 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.900319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.901789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.903356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.903650 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.905084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.909651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:01.931288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1237/2488] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:01.932865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:01.946467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a186-c897-4edd-862a-a3db85272d72 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:01.952379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:01.953230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:01.953786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:01.954149 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] PATH_INFO: `/v3/projects/972f702794014320bb4cb271d92e933c/users/49e35c65172f44e4a4013f84e5567f34/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:01.971702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972f702794014320bb4cb271d92e933c, user_id=49e35c65172f44e4a4013f84e5567f34, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:01.979785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a99226-cf5d-4a02-aaa8-2ddffc965967 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:02.057237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:02.060259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:02.068944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1252/2489] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:01 2022] PUT /identity/v3/projects/972f702794014320bb4cb271d92e933c/users/49e35c65172f44e4a4013f84e5567f34/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:02.080303 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:02.080585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:02.080887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:02.120364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] MFA Rules not processed for user `49e35c65172f44e4a4013f84e5567f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:02.124339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:02.136623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:02.158366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f819d98-5c72-40c8-81c7-22a71df189d4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:02.233259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:02.233724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:02.234001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:02.234279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:02.235762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1253/2490] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:02.237525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:02.248269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:02.249728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:02.250391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:02.330856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:02.333149 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:02.356464 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1238/2491] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:02.362162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d83ee-c010-451e-add5-ef31ae324322 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:02.365804 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:02.366500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:02.366851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:02.367196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:02.387961 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:02.389861 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:02.410898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1254/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:08.375404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:08.377008 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:08.397412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d816fad-b983-4f63-ad54-c3f925752b63 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:08.406569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:08.409345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:08.410891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:08.410891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] PATH_INFO: `/v3/users/eb4c92edcb024e32a549650eb85f6952` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:08.411772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4c92edcb024e32a549650eb85f6952)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:08.415610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:08.629959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4c92edcb024e32a549650eb85f6952'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:08.639659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4c92edcb024e32a549650eb85f6952'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:08.653560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4c92edcb024e32a549650eb85f6952'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:08.654350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4c92edcb024e32a549650eb85f6952'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:08.655016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ccf56dcf-2eb0-4657-9e1f-82c19ac42157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4c92edcb024e32a549650eb85f6952'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:08.779264 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1239/2493] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:08 2022] DELETE /identity/v3/users/eb4c92edcb024e32a549650eb85f6952 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:09.060973 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.072348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b376d-a4ff-4e5b-94c9-ee9d18496fb8 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.132943 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.133207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.133486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.133785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] PATH_INFO: `/v3/projects/ee8baa7d6e294e27ae2fb12eded3709d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.134962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8baa7d6e294e27ae2fb12eded3709d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.137625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.256390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8baa7d6e294e27ae2fb12eded3709d', 'request_id': 'req-2beff894-36a1-415f-949f-fc4bb80583ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:09.259972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.332797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8baa7d6e294e27ae2fb12eded3709d', 'request_id': 'req-2beff894-36a1-415f-949f-fc4bb80583ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:09.338571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8baa7d6e294e27ae2fb12eded3709d', 'request_id': 'req-2beff894-36a1-415f-949f-fc4bb80583ad'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:09.350220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] The token cache is being invalidate because project ee8baa7d6e294e27ae2fb12eded3709d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:09.350509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:09.351160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-2beff894-36a1-415f-949f-fc4bb80583ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:09.352779 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1255/2494] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:09 2022] DELETE /identity/v3/projects/ee8baa7d6e294e27ae2fb12eded3709d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:09.357859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.358170 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.358446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.378859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:09.380053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:09.438416 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1240/2495] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:09.446529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.446816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.447089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.467480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:09.468709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:09.476549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1256/2496] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:09.480749 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.490297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384b25d-94d5-4306-ae46-891e9b9ba3e8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.494963 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.495565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.496831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.496991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.497572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:09.497874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.499477 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.510912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1241/2497] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:09.516104 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.525507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd4078-9778-4a87-aecb-1bfc4e569378 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.532144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.532144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.532144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.532470 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.533275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.535175 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.651081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.652797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1257/2498] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:09.658482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:09.662319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edae4-9015-4de9-b340-8b88bfc74d2b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.732146 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.732766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.733045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.733495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.734142 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.736054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.865958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1242/2499] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:09 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:09.870270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.881682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560954c4-4718-4c8c-95f5-d076cab7ca62 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.886862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.887416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.887719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.887996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.888650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.890353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.900640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1258/2500] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:09.904718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.914628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb14fd7-7c37-4a6c-aa53-976bc34161e7 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.919871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.920458 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.920769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.921042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.921684 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.923355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4997110-6eac-4ce3-ad7b-6368ed90e472 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:09.934781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1243/2501] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:09.938831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:09.955050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978e420-8244-4df1-bbcd-34b26501b11b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:09.955805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:09.956469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:09.956850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:09.957134 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] PATH_INFO: `/v3/projects/1cc77ee3f0f64c4694c816d46fcf8ca9/users/07dd6e3f8e4a49a9a9a43787c46b09ca/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:09.974600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc77ee3f0f64c4694c816d46fcf8ca9, user_id=07dd6e3f8e4a49a9a9a43787c46b09ca, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:09.977270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.123157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1259/2502] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:09 2022] PUT /identity/v3/projects/1cc77ee3f0f64c4694c816d46fcf8ca9/users/07dd6e3f8e4a49a9a9a43787c46b09ca/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:10.134728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.135019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.135341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.162973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] MFA Rules not processed for user `07dd6e3f8e4a49a9a9a43787c46b09ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:10.164161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:10.227537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1244/2503] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:10.232788 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.305062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1f9c-a1b1-43ed-b261-12c8f5213012 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.310001 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.310605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.310883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.311154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.311782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.313722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.481531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1260/2504] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/projects => generated 365 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:10.485608 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.496434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c3186-b648-4d4d-913a-0469ae848338 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.554715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.555432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.555710 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.555983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.556607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.558658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.664506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1245/2505] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:10.668568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.679015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44289ab6-a4e8-4f5c-98a4-5a7f3c037741 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.683880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.684833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.685077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.685365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.686023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.687593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.702889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1261/2506] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.707195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.712386 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.718372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e052-0947-4406-b3e8-3dbd444d18b9 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.722970 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.723639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.723918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.724191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] PATH_INFO: `/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff/users/70f93a26554344219aad40a059807709/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.738800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=340ac4905ba84e2a90d4f6664d70d8ff, user_id=70f93a26554344219aad40a059807709, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.741491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.843135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1246/2507] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff/users/70f93a26554344219aad40a059807709/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.847154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.858395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79707f-1f0c-4c8f-982f-a70da73eb64c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.909394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:10.920621 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.921223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.921519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.921772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.922428 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.924235 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2613b72-7794-4e14-b66a-b9fd157c0c1d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.938200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1262/2508] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:10.944202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:10.956519 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd3c76-2527-4185-828e-f720c1fc98ce admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:10.965776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:10.966676 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.967115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.967470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] PATH_INFO: `/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff/users/70f93a26554344219aad40a059807709/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:10.969750 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340ac4905ba84e2a90d4f6664d70d8ff, user_id=70f93a26554344219aad40a059807709, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:10.973088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:10.983625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1247/2509] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:10 2022] PUT /identity/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff/users/70f93a26554344219aad40a059807709/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:10.994068 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:10.994299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:10.994577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.023499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] MFA Rules not processed for user `70f93a26554344219aad40a059807709`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:11.024823 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:11.093236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1263/2510] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:11.206409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.209482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:11.287545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fcd2-cd6d-4068-9eb9-1c09a9572559 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.291480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.292091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.292390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.292864 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.302804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.304281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.315136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1248/2511] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:11.655741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:11.665447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07938a6c-eed9-4818-871e-ea70976eb83b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:11.669792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:11.670457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:11.670738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:11.671011 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:11.734084 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:11.734947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd7d12-0b0e-42e0-805a-dae86fcda55a service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:11.744826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1264/2512] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:12.314658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.320071 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:12.395396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1051b05-acea-4ac7-9d53-01a98d2b6afd service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:12.400254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:12.400929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.401241 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.401518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] PATH_INFO: `/v3/users/49e35c65172f44e4a4013f84e5567f34` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.402877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e35c65172f44e4a4013f84e5567f34)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:12.404892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:12.704973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e35c65172f44e4a4013f84e5567f34'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:12.712068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e35c65172f44e4a4013f84e5567f34'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:12.723246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:12.725814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e35c65172f44e4a4013f84e5567f34'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:12.727800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e35c65172f44e4a4013f84e5567f34'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:12.727800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e35c65172f44e4a4013f84e5567f34'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:12.838857 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1249/2513] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:12 2022] DELETE /identity/v3/users/49e35c65172f44e4a4013f84e5567f34 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:12.844053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:12.844585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:12.844722 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:12.868430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:12.868430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:12.931712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1265/2514] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:12.942306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:12.943853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:13.007199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c213656-b1de-41bd-a2da-d0571a8550be admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:13.012930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:13.013569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.013946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.014222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.024788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:13.026235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b96b05-cff6-42af-bb3b-b45e14930448 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:13.037133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1250/2515] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:13.659890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:13.661379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:13.677715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a218f08-e536-475f-9ca0-2b5859e75e52 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:13.684845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:13.685702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:13.685906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:13.686188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] PATH_INFO: `/v3/projects/972f702794014320bb4cb271d92e933c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:13.687389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=972f702794014320bb4cb271d92e933c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:13.689732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:13.700496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:13.868764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972f702794014320bb4cb271d92e933c', 'request_id': 'req-f079a247-a88a-460a-97f0-66f08f0b8d85'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:13.999217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972f702794014320bb4cb271d92e933c', 'request_id': 'req-f079a247-a88a-460a-97f0-66f08f0b8d85'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:14.007156 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972f702794014320bb4cb271d92e933c', 'request_id': 'req-f079a247-a88a-460a-97f0-66f08f0b8d85'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:14.021263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] The token cache is being invalidate because project 972f702794014320bb4cb271d92e933c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:14.021541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:14.022210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:14.023745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1266/2516] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:13 2022] DELETE /identity/v3/projects/972f702794014320bb4cb271d92e933c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:19.299525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:19.300020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:19.300119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:19.319047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:19.320257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:19.381163 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1251/2517] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:19.394497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:19.395903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:19.469044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079a247-a88a-460a-97f0-66f08f0b8d85 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:19.474798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:19.475416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:19.475693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:19.475976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:19.485301 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:19.487105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74639f-73f3-40a5-a192-f8f4a1034a51 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:19.496327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1267/2518] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:32:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:32:23.496395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:23.497836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:23.567863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac958f-9357-4c26-be5e-9c5fd574236a None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:23.572624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:23.573290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.573595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.574009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] PATH_INFO: `/v3/users/07dd6e3f8e4a49a9a9a43787c46b09ca` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.575147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07dd6e3f8e4a49a9a9a43787c46b09ca)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:23.577085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:23.813160 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:23.822048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07dd6e3f8e4a49a9a9a43787c46b09ca'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:23.829297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07dd6e3f8e4a49a9a9a43787c46b09ca'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:23.841819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07dd6e3f8e4a49a9a9a43787c46b09ca'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:23.842489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07dd6e3f8e4a49a9a9a43787c46b09ca'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:23.843115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07dd6e3f8e4a49a9a9a43787c46b09ca'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:23.933620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1252/2519] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:23 2022] DELETE /identity/v3/users/07dd6e3f8e4a49a9a9a43787c46b09ca => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:23.937883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:23.938335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:23.938446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:23.958403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:23.959632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.022732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1268/2520] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:24.033219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.035156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.094770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b5ef-792a-47e7-a728-3b5512a30f97 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:24.098335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:24.101017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.101017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.101017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.108791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:24.110219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00376d71-5b97-4179-8177-b7147b615bc0 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:24.119843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1253/2521] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:24.949867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:24.952534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:24.968379 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be9149-820a-49f1-b388-c4599334ae5d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:24.975831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ba38-02e8-4617-a68b-c5e7d8d02618 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.976358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ba38-02e8-4617-a68b-c5e7d8d02618 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.976442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243ba38-02e8-4617-a68b-c5e7d8d02618 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.977953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1254/2522] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 02:32:24.981937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:24.982598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.982876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.983145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] PATH_INFO: `/v3/projects/1cc77ee3f0f64c4694c816d46fcf8ca9` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.987052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:24.987052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:24.987052 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:24.987279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc77ee3f0f64c4694c816d46fcf8ca9)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:24.989160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.007819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] MFA Rules not processed for user `3ce21b04744a41b590dbc8a8640ffc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:25.009144 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:25.017715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1255/2523] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:32:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:25.034107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.035922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:25.047476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b4b74-4214-4c5a-af48-f97ccd4d0d18 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.054005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.054702 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.057120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.057120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.065355 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.073484 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba48fe6-715f-4184-aed1-1f9976dc782e service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.084757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1256/2524] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:25.155347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.177735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2751c5-81ee-4aca-9843-f66e01a50003 None None] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.178018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2751c5-81ee-4aca-9843-f66e01a50003 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.178319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2751c5-81ee-4aca-9843-f66e01a50003 None None] PATH_INFO: `/` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.179678 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1257/2525] 10.222.0.27 () {54 vars in 921 bytes} [Mon Aug 8 02:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 02:32:25.185358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.185734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.186122 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.206669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] MFA Rules not processed for user `fcfff7acf53f4d779549a9688117f05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:25.207851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:25.271881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc77ee3f0f64c4694c816d46fcf8ca9', 'request_id': 'req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.280475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1258/2526] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Aug 8 02:32:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:25.291105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.301111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6bc92-b203-474a-ac06-6e3376e6ecc3 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.327215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.349435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.349968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.350244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.350515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.416175 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.418476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.429216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1259/2527] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:25.438678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc77ee3f0f64c4694c816d46fcf8ca9', 'request_id': 'req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.447661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc77ee3f0f64c4694c816d46fcf8ca9', 'request_id': 'req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.457288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] The token cache is being invalidate because project 1cc77ee3f0f64c4694c816d46fcf8ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:25.457578 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.458240 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.459821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1269/2528] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:24 2022] DELETE /identity/v3/projects/1cc77ee3f0f64c4694c816d46fcf8ca9 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:25.463994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:25.527814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec864a-2767-448c-ab04-7215019e1d82 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:25.532399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:25.533051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:25.533332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:25.533695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] PATH_INFO: `/v3/users/70f93a26554344219aad40a059807709` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:25.534838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f93a26554344219aad40a059807709)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:25.536783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:25.854862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f93a26554344219aad40a059807709'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.862418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f93a26554344219aad40a059807709'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.873556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:25.874758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f93a26554344219aad40a059807709'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.875442 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f93a26554344219aad40a059807709'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.876045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f93a26554344219aad40a059807709'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:25.999281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1260/2529] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:25 2022] DELETE /identity/v3/users/70f93a26554344219aad40a059807709 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:26.218482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.219913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.233441 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2450a-1eb3-41cf-b13d-dce1ee2fb143 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.287918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.288619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.289189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.289666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.292028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.362026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.366995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf76c6-21fb-4339-8fb6-6ef428f92cf5 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.374060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1270/2530] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:26.476061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.477626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.497693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2efd06-a6c7-4878-9a61-136b4e5363bb admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.502808 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.504048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.504337 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.504819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] PATH_INFO: `/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.506157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] RBAC: Authorizing `identity:delete_project(project_id=340ac4905ba84e2a90d4f6664d70d8ff)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.508127 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.639472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340ac4905ba84e2a90d4f6664d70d8ff', 'request_id': 'req-df254f17-526c-4bec-93e1-f58fb53f332f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:26.738961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340ac4905ba84e2a90d4f6664d70d8ff', 'request_id': 'req-df254f17-526c-4bec-93e1-f58fb53f332f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:26.746842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340ac4905ba84e2a90d4f6664d70d8ff', 'request_id': 'req-df254f17-526c-4bec-93e1-f58fb53f332f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:26.761439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] The token cache is being invalidate because project 340ac4905ba84e2a90d4f6664d70d8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:26.761781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:26.763132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-df254f17-526c-4bec-93e1-f58fb53f332f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:26.768154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1261/2531] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:26 2022] DELETE /identity/v3/projects/340ac4905ba84e2a90d4f6664d70d8ff => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:26.769877 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.770230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.770574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.800495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.800495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.854180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:26.870096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1271/2532] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.872367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.873047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.873047 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.894732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:26.896212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:26.905910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1262/2533] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:26.911545 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.922913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835aa7-7a20-4bbb-800e-a208a36694a0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.928507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.929174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.929458 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.929733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.930386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:26.930691 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.933976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:26.945737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1272/2534] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:26.950738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:26.960888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363775ae-991b-4b07-92e7-80dff9d51535 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:26.966602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:26.968151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:26.968151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:26.968151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:26.968547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:26.970689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.091275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1263/2535] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:26 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:27.091894 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.103668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961895f-50b0-46e4-89bf-307dfc4366bd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.165827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.166446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.166721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.166990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.168274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.170927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.273972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1273/2536] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:27.278791 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.289541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617098c-b175-46a7-bfff-28840dcfb811 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.295089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.295772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.296141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.296483 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.297238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.298915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.309468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1264/2537] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:27.313592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.324184 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce503-d415-4476-9fe7-053183fa7a91 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.329057 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.329663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.329945 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.330223 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.330909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.336083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f6ce8-9ee9-4e32-aa0e-6fff7cbd5f46 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.344745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1274/2538] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:27.348909 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.360494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5870f-f4cf-4715-855f-ade1a881de8f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.368454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.374511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.374511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.374511 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] PATH_INFO: `/v3/projects/0f55bb028f9a4691a7104d771a3016e8/users/e93643ed6bf94e088a431b545f2a8cd8/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.384858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f55bb028f9a4691a7104d771a3016e8, user_id=e93643ed6bf94e088a431b545f2a8cd8, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.387775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.524466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1265/2539] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:27 2022] PUT /identity/v3/projects/0f55bb028f9a4691a7104d771a3016e8/users/e93643ed6bf94e088a431b545f2a8cd8/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:27.535269 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.535552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.535822 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.564980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] MFA Rules not processed for user `e93643ed6bf94e088a431b545f2a8cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:27.566339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:27.637249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1275/2540] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:27.641896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.703817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56329a-c225-4a9c-b020-b5bbc9e527b1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.708383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.709090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.709304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.709573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.710195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.712087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:27.729042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:27.829129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1266/2541] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:27.832863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:27.853237 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56050b22-f71c-462b-9e54-bfbbc6c95ff6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:27.945049 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:27.958318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:27.959438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:27.959758 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:27.960147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:27.961323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:27.966386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.073220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1276/2542] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:27 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:28.074561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:28.085002 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396cca9-8471-4ea9-a8c8-9799d61ef465 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.090359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.090967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.091249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.091522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.092253 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.093945 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.105566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1267/2543] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:28.110004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:28.119957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5243c27-5f49-445b-91dd-a1735f8e8e65 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.125312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.125963 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.126239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.126509 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] PATH_INFO: `/v3/projects/41b813d5affd41a2aa95448c0278d5e2/users/133794c97f60455e883c99f0974a2409/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.142378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b813d5affd41a2aa95448c0278d5e2, user_id=133794c97f60455e883c99f0974a2409, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.145431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.234617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1277/2544] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:28 2022] PUT /identity/v3/projects/41b813d5affd41a2aa95448c0278d5e2/users/133794c97f60455e883c99f0974a2409/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:28.238586 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:28.249046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95148c6b-8757-4dc9-b837-4c9339308daa admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.325819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.326584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.326874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.327151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.328135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.330176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c16e7-4d10-44b7-b12a-fc18c9c35f1a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.342450 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1268/2545] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:28.346305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:28.358474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fe11a-de4d-4523-ac70-68242071f38a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.364735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.365674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.366122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.366498 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] PATH_INFO: `/v3/projects/41b813d5affd41a2aa95448c0278d5e2/users/133794c97f60455e883c99f0974a2409/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.368607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41b813d5affd41a2aa95448c0278d5e2, user_id=133794c97f60455e883c99f0974a2409, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.371633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.383113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1278/2546] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:28 2022] PUT /identity/v3/projects/41b813d5affd41a2aa95448c0278d5e2/users/133794c97f60455e883c99f0974a2409/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:28.394197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.394491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.394767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.425620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] MFA Rules not processed for user `133794c97f60455e883c99f0974a2409`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:28.426841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:28.541196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1269/2547] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:28.549381 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:28.551153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:28.632643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ff370-30fe-4d4d-bf62-a8df38c9fe4c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:28.637959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:28.638585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:28.638869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:28.639229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:28.655222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:28.656694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:28.679607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1279/2548] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:28.996025 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:29.005844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ff72c-d5f3-4615-bb8f-d6de0a3ead96 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:29.009670 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:29.010267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:29.010544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:29.010814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:29.068629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:29.070196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:29.080110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1270/2549] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:30.822856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:30.825117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:30.894635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912b81-15f2-4b5c-b030-0d86d5fefda7 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:30.902716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:30.902716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:30.902716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:30.902716 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] PATH_INFO: `/v3/users/578cd837d9fe442587f8c6c8528334c0` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:30.903702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578cd837d9fe442587f8c6c8528334c0)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:30.905650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:31.138760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578cd837d9fe442587f8c6c8528334c0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.143334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:31.146285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578cd837d9fe442587f8c6c8528334c0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.159396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578cd837d9fe442587f8c6c8528334c0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.160099 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578cd837d9fe442587f8c6c8528334c0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.160829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-09a96304-5753-46d7-a63c-5fbee3849468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578cd837d9fe442587f8c6c8528334c0'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.264468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1280/2550] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:30 2022] DELETE /identity/v3/users/578cd837d9fe442587f8c6c8528334c0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:31.557990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:31.564451 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:31.570413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96965b-1df6-45b4-9ebf-8d6a384132b6 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:31.630732 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:31.631349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:31.631628 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:31.631899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] PATH_INFO: `/v3/projects/ca29b2b9a59d4826bfd25f4dcfa9cffe` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:31.633280 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca29b2b9a59d4826bfd25f4dcfa9cffe)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:31.635231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:31.815547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'request_id': 'req-45952daf-93c9-4967-a056-afb010ccff7f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.902407 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'request_id': 'req-45952daf-93c9-4967-a056-afb010ccff7f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.908798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca29b2b9a59d4826bfd25f4dcfa9cffe', 'request_id': 'req-45952daf-93c9-4967-a056-afb010ccff7f'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.921216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] The token cache is being invalidate because project ca29b2b9a59d4826bfd25f4dcfa9cffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:31.921558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.922200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-45952daf-93c9-4967-a056-afb010ccff7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:31.924065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1271/2551] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:31 2022] DELETE /identity/v3/projects/ca29b2b9a59d4826bfd25f4dcfa9cffe => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:31.929050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dfd28-9f10-4585-995f-97086af6530b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:31.929438 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dfd28-9f10-4585-995f-97086af6530b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:31.929579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449dfd28-9f10-4585-995f-97086af6530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:31.949884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-449dfd28-9f10-4585-995f-97086af6530b None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:31.951103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-449dfd28-9f10-4585-995f-97086af6530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.011294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1281/2552] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:32.019085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.019372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.019656 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.039491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:32.040725 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.049301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1272/2553] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:32.053468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dfd28-9f10-4585-995f-97086af6530b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.063093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449dfd28-9f10-4585-995f-97086af6530b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.067703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.068238 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.068528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.068827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.069471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:32.069775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.071353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.083940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1282/2554] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:32:32.088018 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.097872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105f86b-5b67-4276-8a48-3337fc2b1ab0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.102409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.103003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.103623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.103623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.104135 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.106227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.192561 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1273/2555] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:32 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:32.196367 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.207941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fc23-2dd3-46d6-8b2b-b228182f5229 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.265685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.266305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.266588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.266863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.267505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.269537 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.355718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1283/2556] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:32 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:32.360041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.370441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487ec8d-57c9-41fd-aab6-67e199992ce3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.375072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.375669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.375948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.376216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.376893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.378541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.389528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1274/2557] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:32.393592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.404559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e561-b523-4aa2-a8ad-df5bd587d26e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.409224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.409815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.410114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.410389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.411033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.412613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b619a-3d1e-436b-9818-992d1234140a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.425127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1284/2558] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:32.430639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.438915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1c640-a122-4f9e-ae30-0d68b3aabd28 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.443795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.444987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.444987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.445222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] PATH_INFO: `/v3/projects/dbe03b54834641bfbed01df88f536154/users/9a0912f1d353457c9bee96549f1af5a3/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.461362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe03b54834641bfbed01df88f536154, user_id=9a0912f1d353457c9bee96549f1af5a3, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.465120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.550724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1275/2559] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:32 2022] PUT /identity/v3/projects/dbe03b54834641bfbed01df88f536154/users/9a0912f1d353457c9bee96549f1af5a3/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:32.561911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.562199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.562472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.601025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] MFA Rules not processed for user `9a0912f1d353457c9bee96549f1af5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:32.603110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.631123 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:32.675554 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1285/2560] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:32.687984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.689758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.716249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:32.718469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:32.738878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:32.784975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee81c2-2ee3-4bec-b293-5c1969d0299e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.789780 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3d0ef-ea66-4198-be4c-9cedd52ef69f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:32.791095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.791699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.791975 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.792809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.795259 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:32.795913 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:32.796188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:32.796460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] PATH_INFO: `/v3/users/e93643ed6bf94e088a431b545f2a8cd8` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:32.797658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93643ed6bf94e088a431b545f2a8cd8)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.799567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.800623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:32.802248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbe575-6948-4028-8424-38f74f7c2514 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:32.813020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1276/2561] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:33.055116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93643ed6bf94e088a431b545f2a8cd8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.062652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93643ed6bf94e088a431b545f2a8cd8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.074754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93643ed6bf94e088a431b545f2a8cd8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.075451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93643ed6bf94e088a431b545f2a8cd8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.076072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93643ed6bf94e088a431b545f2a8cd8'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:33.195316 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1286/2562] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:32 2022] DELETE /identity/v3/users/e93643ed6bf94e088a431b545f2a8cd8 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:33.195468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.195468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.195468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.211876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:33.213140 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.270520 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1277/2563] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:33.281144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:33.282790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.335555 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:33.348473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c876d3b-f623-42f6-b146-fb94e067bab9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:33.353277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:33.354614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.354614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.354872 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.365827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:33.367306 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:33.379129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1287/2564] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:33.864199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:33.865759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:33.879009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1080c91-9708-43f3-b639-05a9d0ee5a1c None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:33.884638 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:33.885345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:33.885706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:33.886022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] PATH_INFO: `/v3/projects/0f55bb028f9a4691a7104d771a3016e8` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:33.887424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f55bb028f9a4691a7104d771a3016e8)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:33.889478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.018361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f55bb028f9a4691a7104d771a3016e8', 'request_id': 'req-57f10d2a-72d9-40eb-b809-8d722a7a9ded'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.115358 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f55bb028f9a4691a7104d771a3016e8', 'request_id': 'req-57f10d2a-72d9-40eb-b809-8d722a7a9ded'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.121486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f55bb028f9a4691a7104d771a3016e8', 'request_id': 'req-57f10d2a-72d9-40eb-b809-8d722a7a9ded'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.132473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] The token cache is being invalidate because project 0f55bb028f9a4691a7104d771a3016e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:34.132787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.133427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.134924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1278/2565] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:33 2022] DELETE /identity/v3/projects/0f55bb028f9a4691a7104d771a3016e8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:34.139109 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:34.149096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc39f2-e1cd-42ce-bbcb-3383f4f081a5 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:34.206692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:34.207313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:34.207592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:34.207864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] PATH_INFO: `/v3/users/133794c97f60455e883c99f0974a2409` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:34.209003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=133794c97f60455e883c99f0974a2409)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:34.210909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:34.473434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133794c97f60455e883c99f0974a2409'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.480767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133794c97f60455e883c99f0974a2409'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.492473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133794c97f60455e883c99f0974a2409'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.493246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133794c97f60455e883c99f0974a2409'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.498181 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aa96db9e-cada-4f2b-ad80-ecad46e20cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133794c97f60455e883c99f0974a2409'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:34.579892 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1288/2566] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:34 2022] DELETE /identity/v3/users/133794c97f60455e883c99f0974a2409 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:35.064746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.066697 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.085671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f10d2a-72d9-40eb-b809-8d722a7a9ded admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.129560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:35.151946 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.152645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.152982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.153244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] PATH_INFO: `/v3/projects/41b813d5affd41a2aa95448c0278d5e2` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.154430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=41b813d5affd41a2aa95448c0278d5e2)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.156310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.286653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41b813d5affd41a2aa95448c0278d5e2', 'request_id': 'req-27944c96-e4be-499d-93c4-fd208d139e8a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:35.395521 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41b813d5affd41a2aa95448c0278d5e2', 'request_id': 'req-27944c96-e4be-499d-93c4-fd208d139e8a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:35.401300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41b813d5affd41a2aa95448c0278d5e2', 'request_id': 'req-27944c96-e4be-499d-93c4-fd208d139e8a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:35.413394 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] The token cache is being invalidate because project 41b813d5affd41a2aa95448c0278d5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:35.413688 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:35.414316 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-27944c96-e4be-499d-93c4-fd208d139e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:35.415781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1279/2567] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:35 2022] DELETE /identity/v3/projects/41b813d5affd41a2aa95448c0278d5e2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:35.421848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.422330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.422450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.441530 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:35.442732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.502524 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1289/2568] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:35.512061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.512344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.512644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.534105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:35.535289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:35.545255 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1280/2569] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:35.549838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.559821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e3ba3-3043-46e3-ae11-79060d751ed4 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.564439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.565079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.565377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.565636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.566277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:35.566581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.568170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.569715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:35.580587 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1290/2570] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:35.584984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.595663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09695339-e381-466c-9fe6-20c32d5f2f37 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.600593 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.601289 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.601568 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.601843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.602536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.604447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.759997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1281/2571] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:35 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:35.764517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.774864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e0d01-efac-4838-9b1e-6e4ce48a3466 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.833115 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.833729 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.834010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.834285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.834951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.836845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.962034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1291/2572] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:35 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:35.967224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:35.978031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f746118-6f09-4288-8f95-eeb9c4acbd6a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:35.983003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:35.983594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:35.983875 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:35.984148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:35.984830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:35.986410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:35.997136 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1282/2573] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:36.001278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.011571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c142b-aa6b-4669-b262-84540a567266 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.016548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.017172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.017450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.017721 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.018353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.019993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.031574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1292/2574] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:36.035559 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.045755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e41d2b-7ede-4e6f-bc59-9cd782d915b1 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.050388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.051030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.051313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.051591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] PATH_INFO: `/v3/projects/d5d6756a54254bfd8a12e0501d27c8b0/users/f31f791f6c4a41df8f326388aee73c32/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.068225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d6756a54254bfd8a12e0501d27c8b0, user_id=f31f791f6c4a41df8f326388aee73c32, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.071204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a24ce7-d628-4bdb-9d4e-0a90d9e998c9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.167850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.169598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.172443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1283/2575] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:36 2022] PUT /identity/v3/projects/d5d6756a54254bfd8a12e0501d27c8b0/users/f31f791f6c4a41df8f326388aee73c32/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:36.182227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c1eba-66b7-4d6a-8d9b-3f69f97f7ef9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.185565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.185853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.186131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.216641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] MFA Rules not processed for user `f31f791f6c4a41df8f326388aee73c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:36.218284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.242754 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.243385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.243661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.244003 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] PATH_INFO: `/v3/users/9a0912f1d353457c9bee96549f1af5a3` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.245202 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0912f1d353457c9bee96549f1af5a3)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.247581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.272811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:36.282594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1284/2576] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:36.287205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.297390 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ad14a-a29f-4e1d-bbc1-58fa6b3a58c5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.302274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.302862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.303137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.303437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.304055 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.306029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.461058 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1285/2577] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:36 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:36.465051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.526375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9efc5-645f-490b-bf69-611b4a58eccf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.529724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0912f1d353457c9bee96549f1af5a3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.531411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.532046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.532330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.532606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.533389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.535434 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.536775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0912f1d353457c9bee96549f1af5a3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.550142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0912f1d353457c9bee96549f1af5a3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.550842 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0912f1d353457c9bee96549f1af5a3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.554167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-95b13321-b995-4dd1-b932-61a9a485636f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0912f1d353457c9bee96549f1af5a3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:36.554167 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:36.689101 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1293/2578] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:36 2022] DELETE /identity/v3/users/9a0912f1d353457c9bee96549f1af5a3 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:36.689886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1286/2579] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:36 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:36.693559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.693847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.694119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.695925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.698473 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.711380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48725b3-d47f-479c-9baa-585e11a9f32b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.716970 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:36.718187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.775735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.776437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.776720 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.777690 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.778350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.779955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.781793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1294/2580] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:36.791610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1287/2581] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:36.796062 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.797424 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.802277 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:36.808809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1d6d3-e471-4fc2-9528-f4010ada7c58 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.813288 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.813956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.814239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.814512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] PATH_INFO: `/v3/projects/5407b879cb6b4d22995f8005adf220ea/users/8777e525021d4453b1a416e2860d306a/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.831581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5407b879cb6b4d22995f8005adf220ea, user_id=8777e525021d4453b1a416e2860d306a, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.834502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.865356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1ac2c-5f9b-487e-bb4d-7e75491d9600 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:36.869126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:36.869813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:36.870126 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:36.870987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:36.880830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:36.882266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:36.887321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:36.892019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1288/2582] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:36.950719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1295/2583] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:36 2022] PUT /identity/v3/projects/5407b879cb6b4d22995f8005adf220ea/users/8777e525021d4453b1a416e2860d306a/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:36.954788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:36.965420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aae8a-f75b-4e08-a154-857502d073df service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:37.036353 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:37.037026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.037329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.037602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.038243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:37.039816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92c55e-5ca6-415a-8fac-9eb5b883e0d7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:37.054117 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1289/2584] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:37.055383 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:37.066212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e1aa-6864-4470-bf0b-133aa413a6f2 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:37.071075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:37.071985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.072262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.072532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] PATH_INFO: `/v3/projects/5407b879cb6b4d22995f8005adf220ea/users/8777e525021d4453b1a416e2860d306a/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.075008 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5407b879cb6b4d22995f8005adf220ea, user_id=8777e525021d4453b1a416e2860d306a, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:37.077673 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:37.088462 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1296/2585] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:37 2022] PUT /identity/v3/projects/5407b879cb6b4d22995f8005adf220ea/users/8777e525021d4453b1a416e2860d306a/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:37.107522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.107522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.107522 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.133032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] MFA Rules not processed for user `8777e525021d4453b1a416e2860d306a`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:37.134278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.228093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1290/2586] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:37.236617 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:37.239053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.309791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698fe8-bc08-4451-992b-0c6f1661cea2 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:37.313632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:37.314353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.314631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.314903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.325368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:37.326885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:37.337227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1297/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:37.405921 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:37.417717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:37.419211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.486745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbc86f-9dfc-416c-b1ae-9ad7706be758 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:37.491370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:37.492000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.492282 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.492556 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] PATH_INFO: `/v3/users/f31f791f6c4a41df8f326388aee73c32` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.493731 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31f791f6c4a41df8f326388aee73c32)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:37.495878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:37.499056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:37.503437 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6e0ce-f274-4482-b863-bb9324d8a489 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:37.508210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:37.508871 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.509178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.509789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] PATH_INFO: `/v3/projects/dbe03b54834641bfbed01df88f536154` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.510694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe03b54834641bfbed01df88f536154)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:37.513154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:37.521771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:37.789141 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31f791f6c4a41df8f326388aee73c32'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.789873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe03b54834641bfbed01df88f536154', 'request_id': 'req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.796506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31f791f6c4a41df8f326388aee73c32'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.808707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31f791f6c4a41df8f326388aee73c32'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.809595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31f791f6c4a41df8f326388aee73c32'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.810234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-bb0f818e-4af7-4ed8-b6e7-cbc50b718d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31f791f6c4a41df8f326388aee73c32'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.875490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe03b54834641bfbed01df88f536154', 'request_id': 'req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.882136 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe03b54834641bfbed01df88f536154', 'request_id': 'req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.894375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] The token cache is being invalidate because project dbe03b54834641bfbed01df88f536154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:37.894671 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.895322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-e7b0bcf2-379f-4d30-9e0d-0f7012d4186b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:37.896843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1298/2588] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:37 2022] DELETE /identity/v3/projects/dbe03b54834641bfbed01df88f536154 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:37.901455 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed432-7619-4db6-9ced-9421bba7d9a8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.901771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed432-7619-4db6-9ced-9421bba7d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.902074 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed432-7619-4db6-9ced-9421bba7d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.921275 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-442ed432-7619-4db6-9ced-9421bba7d9a8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:37.922486 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-442ed432-7619-4db6-9ced-9421bba7d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.943838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1291/2589] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:37 2022] DELETE /identity/v3/users/f31f791f6c4a41df8f326388aee73c32 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:37.948576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.949466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.949740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:37.970441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:37.971636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:37.983243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1299/2590] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:37.992607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:37.992959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:37.993351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.014552 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:38.015889 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.031745 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1292/2591] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:38.048041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.049399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.077571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1300/2592] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:38.081734 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.091172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f1684-8d87-4496-bcfd-80460a6421bb None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.095876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.096460 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.096806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.097076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.097719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:38.098025 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.099614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.111273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddef-e1c6-43ee-8f55-624bcfe58569 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.112135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1301/2593] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:32:38.115032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.115778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.116062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.116334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.116453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.126757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.128037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bbc35-1c6c-4ecd-ba32-a01a3e85a0d8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.128165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.133258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.134233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.134233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.134507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.135178 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.137520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.138555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1293/2594] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:38.233456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1302/2595] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:38 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:38.235554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.246249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d146a-2495-4150-958a-950c533e57a6 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.248060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.288804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:38.303689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.304290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.304566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.304874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.305500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.307398 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.431281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1294/2596] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:38 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:38.435639 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.446854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b4105-50a5-4587-8ead-d6c11afdfa50 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.451985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.452594 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.452911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.453341 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.454124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.455835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.468813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1303/2597] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:38.472780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.484275 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf360926-c870-4d38-898f-10c6b6f9f85e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.489169 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.489803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.490083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.490356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.491073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.492630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.502847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1295/2598] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:38.508901 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.517860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd77e66-3380-44a0-adf7-5a0556f37b15 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.522466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:38.523145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.523424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.523724 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] PATH_INFO: `/v3/projects/b63c63fe631f461ea8a0f4fdcd5eb367/users/f992231995a0497f874af1c54dcd5c86/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.538835 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63c63fe631f461ea8a0f4fdcd5eb367, user_id=f992231995a0497f874af1c54dcd5c86, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:38.541834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ed19-853f-4ea4-a1c7-d6cf5a62381a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:38.934684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1304/2599] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:38 2022] PUT /identity/v3/projects/b63c63fe631f461ea8a0f4fdcd5eb367/users/f992231995a0497f874af1c54dcd5c86/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:38.937421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:38.941895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:38.944883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:38.945170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:38.946328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:38.959863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137f535-a902-4ee6-b543-02ccb8200661 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:38.981487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] MFA Rules not processed for user `f992231995a0497f874af1c54dcd5c86`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:38.982762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.021977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.022904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.023360 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.023635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] PATH_INFO: `/v3/projects/d5d6756a54254bfd8a12e0501d27c8b0` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.025430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d6756a54254bfd8a12e0501d27c8b0)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.027925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.050640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1305/2600] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:39.055030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.067326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73077-6c70-4d06-968c-6a23e6750661 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.072434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.073153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.073660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.074105 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.074839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.076852 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.187384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d6756a54254bfd8a12e0501d27c8b0', 'request_id': 'req-4fd37551-02ac-4c3e-9809-40f5f37892f4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.216326 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1306/2601] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:39 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:39.220321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.283942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa00f6c-5e89-4526-be28-1d137eef1d2a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.288856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.289527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.289805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.290070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.290685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.293353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.304406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d6756a54254bfd8a12e0501d27c8b0', 'request_id': 'req-4fd37551-02ac-4c3e-9809-40f5f37892f4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.310700 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d6756a54254bfd8a12e0501d27c8b0', 'request_id': 'req-4fd37551-02ac-4c3e-9809-40f5f37892f4'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.322447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] The token cache is being invalidate because project d5d6756a54254bfd8a12e0501d27c8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:39.322737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.323363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.325033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1296/2602] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:38 2022] DELETE /identity/v3/projects/d5d6756a54254bfd8a12e0501d27c8b0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:39.328896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.330226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.383733 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:39.397421 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37551-02ac-4c3e-9809-40f5f37892f4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.403621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.404257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.404539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.406347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] PATH_INFO: `/v3/users/8777e525021d4453b1a416e2860d306a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.409339 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8777e525021d4453b1a416e2860d306a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.412073 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.452253 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1307/2603] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:39 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:39.455246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.462587 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.477843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d06cd6-b3a8-43af-bb16-84d97a9872bd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.485169 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.485794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.486079 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.486353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.487055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.488738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.499532 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1308/2604] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:39.504197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.512875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:39.518810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7dd09-8484-4483-92c7-f464964a99f6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.524557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.525312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.525603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.525881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] PATH_INFO: `/v3/projects/acda4529648b41ae94fabf0bba2538be/users/2ee0a86346ff4257a8e33e1c523b4f4b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.545336 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acda4529648b41ae94fabf0bba2538be, user_id=2ee0a86346ff4257a8e33e1c523b4f4b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.548232 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.614872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8777e525021d4453b1a416e2860d306a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.622229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1309/2605] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:39 2022] PUT /identity/v3/projects/acda4529648b41ae94fabf0bba2538be/users/2ee0a86346ff4257a8e33e1c523b4f4b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:39.622600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8777e525021d4453b1a416e2860d306a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.626247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.636746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8777e525021d4453b1a416e2860d306a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.637615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8777e525021d4453b1a416e2860d306a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.639944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8777e525021d4453b1a416e2860d306a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:39.692736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cf91f-83c5-4e8d-874c-46a701083e47 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.697740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.698348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.699146 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.699408 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.700098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.702515 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.713609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1310/2606] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:32:39.718366 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.720092 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.733425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b8ed7-0578-4f37-99fe-02377223df56 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.738206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.738860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.739140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.739414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] PATH_INFO: `/v3/projects/acda4529648b41ae94fabf0bba2538be/users/2ee0a86346ff4257a8e33e1c523b4f4b/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.741343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acda4529648b41ae94fabf0bba2538be, user_id=2ee0a86346ff4257a8e33e1c523b4f4b, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.743962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff7ca0-0222-4497-becb-6e9c6ce4ead4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.754356 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1311/2607] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:39 2022] PUT /identity/v3/projects/acda4529648b41ae94fabf0bba2538be/users/2ee0a86346ff4257a8e33e1c523b4f4b/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:39.771789 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.772071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.772346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.793324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1297/2608] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:39 2022] DELETE /identity/v3/users/8777e525021d4453b1a416e2860d306a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:39.807162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] MFA Rules not processed for user `2ee0a86346ff4257a8e33e1c523b4f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:39.808988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.885490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1312/2609] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:39.900877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:39.900877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:39.958867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715abd4a-f8d3-4507-a93c-b6205d99cd5b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:39.962501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:39.963109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:39.963388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:39.963659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:39.973877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:39.975295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ff413-900d-41c2-a39e-76c3706628ee service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:39.985495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1298/2610] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:40.224131 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.231312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.280274 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:40.296327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb92ecd-55fe-42cf-8c54-72ed046e9bd0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.306457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.307129 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.307413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.307687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] PATH_INFO: `/v3/projects/5407b879cb6b4d22995f8005adf220ea` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.309674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5407b879cb6b4d22995f8005adf220ea)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.316643 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.468685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5407b879cb6b4d22995f8005adf220ea', 'request_id': 'req-8330c4b4-7c06-4720-95ce-a3adb5eb3342'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.586908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5407b879cb6b4d22995f8005adf220ea', 'request_id': 'req-8330c4b4-7c06-4720-95ce-a3adb5eb3342'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.601208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5407b879cb6b4d22995f8005adf220ea', 'request_id': 'req-8330c4b4-7c06-4720-95ce-a3adb5eb3342'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.613296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] The token cache is being invalidate because project 5407b879cb6b4d22995f8005adf220ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:40.613611 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.614285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8330c4b4-7c06-4720-95ce-a3adb5eb3342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:40.615899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1313/2611] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:40 2022] DELETE /identity/v3/projects/5407b879cb6b4d22995f8005adf220ea => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:40.621691 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.622220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.622220 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.642086 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:40.643287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.709631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1299/2612] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:40.719685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.720041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.720597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.743795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:40.749152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:40.755384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1314/2613] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:40.759327 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.769860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f7da1-3d10-4c1e-bfd5-58491f7af09e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.787711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.787711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.787711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.787711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.788107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:40.788107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.788107 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.803538 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1300/2614] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:40.804956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.814883 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c33834-f42a-422e-9e6a-d29e7e33efe2 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:40.819674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:40.821971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:40.821971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:40.821971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:40.821971 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:40.824393 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:40.947087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1315/2615] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:40.947886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:40.953420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:40.958831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d757ef-318b-47b4-b8aa-d5b9ebac61e5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.016619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.017246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.017518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.017789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.018422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.020313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.099365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1301/2616] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:40 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:32:41.103405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.113873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4975-f5ac-4dd4-9361-20940722e5f9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.118564 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.119158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.126111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.126111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.126111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.126111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.132030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1316/2617] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:41.135814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.146461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdaf6-6cea-4584-9b6a-986f07b94605 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.151092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.152790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.153074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.153347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.153990 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.155564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906f191-264e-4b60-a8cf-5a041abd7ef9 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.166076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1302/2618] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:41.170343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.181161 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3202772-c154-4a1a-995d-9aa56cf43b4d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.188205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.189188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.189497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.189771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] PATH_INFO: `/v3/projects/414b6a313558425cbd3ed5c6da5405a9/users/ffcf7b8ddd454d9fa3b32f86906cedd5/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.210466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414b6a313558425cbd3ed5c6da5405a9, user_id=ffcf7b8ddd454d9fa3b32f86906cedd5, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.213523 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.293958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1317/2619] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:41 2022] PUT /identity/v3/projects/414b6a313558425cbd3ed5c6da5405a9/users/ffcf7b8ddd454d9fa3b32f86906cedd5/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:32:41.299429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9332-18d0-4450-8e93-587b046dd9e3 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.299740 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9332-18d0-4450-8e93-587b046dd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.300013 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f9332-18d0-4450-8e93-587b046dd9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.327439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-931f9332-18d0-4450-8e93-587b046dd9e3 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:41.327439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-931f9332-18d0-4450-8e93-587b046dd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.395777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1303/2620] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:41.406870 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:41.408590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:41.477989 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa5b6b-bccb-4600-b039-3dd3384e74c9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:41.482580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:41.483190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:41.483467 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:41.483736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:41.493144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:41.494576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:41.504290 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1318/2621] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:42.891567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:42.896214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:42.896214 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:42.923505 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] MFA Rules not processed for user `ffcf7b8ddd454d9fa3b32f86906cedd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:42.924747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:42.999801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1304/2622] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:43.010718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.012947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:43.033572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:43.075815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b678a-a1f9-433b-a91f-500be550b964 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.079623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.080265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.080512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.080813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.089598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.090967 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120767-d77c-4242-8557-1426e1338226 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.099785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1319/2623] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:43.207319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.208925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:43.222482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2874c-0f56-4d64-aeef-1da12c712b9e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.227181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.227798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.228074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.228379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] PATH_INFO: `/v3/users/f992231995a0497f874af1c54dcd5c86` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.229485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f992231995a0497f874af1c54dcd5c86)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.231728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.236429 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:43.440523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f992231995a0497f874af1c54dcd5c86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:43.447372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f992231995a0497f874af1c54dcd5c86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:43.459095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f992231995a0497f874af1c54dcd5c86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:43.459817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f992231995a0497f874af1c54dcd5c86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:43.460417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f992231995a0497f874af1c54dcd5c86'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:43.564624 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1305/2624] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:43 2022] DELETE /identity/v3/users/f992231995a0497f874af1c54dcd5c86 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:43.569100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.569384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.569654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.589141 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:43.590334 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:43.648349 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1320/2625] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:43.658820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:43.660204 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:43.717845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf852e8-1268-423f-8b20-a3f969749b9e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:43.721404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:43.722000 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:43.722278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:43.722552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:43.733200 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:43.734671 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:43.744235 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1306/2626] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:44.225784 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.227260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:44.241152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9a95b-3cf6-4518-b1dd-2b297fc637ba None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.246521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.247152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.247551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.247824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] PATH_INFO: `/v3/projects/b63c63fe631f461ea8a0f4fdcd5eb367` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.249623 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63c63fe631f461ea8a0f4fdcd5eb367)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.251577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.374189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63c63fe631f461ea8a0f4fdcd5eb367', 'request_id': 'req-5e47f406-6a30-435a-a3cc-0a0214f8ab78'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.470020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63c63fe631f461ea8a0f4fdcd5eb367', 'request_id': 'req-5e47f406-6a30-435a-a3cc-0a0214f8ab78'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.476090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63c63fe631f461ea8a0f4fdcd5eb367', 'request_id': 'req-5e47f406-6a30-435a-a3cc-0a0214f8ab78'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.488315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] The token cache is being invalidate because project b63c63fe631f461ea8a0f4fdcd5eb367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:44.488603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.489277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-5e47f406-6a30-435a-a3cc-0a0214f8ab78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.490776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1321/2627] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:44 2022] DELETE /identity/v3/projects/b63c63fe631f461ea8a0f4fdcd5eb367 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:44.495059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:44.504907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800131d-4c90-4982-b16f-41b85f8f1699 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:44.561482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:44.562249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.562528 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.562879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] PATH_INFO: `/v3/users/2ee0a86346ff4257a8e33e1c523b4f4b` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.564003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee0a86346ff4257a8e33e1c523b4f4b)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:44.565950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:44.726432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:44.858374 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee0a86346ff4257a8e33e1c523b4f4b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.865096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee0a86346ff4257a8e33e1c523b4f4b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.877591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee0a86346ff4257a8e33e1c523b4f4b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.878320 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee0a86346ff4257a8e33e1c523b4f4b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.878925 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee0a86346ff4257a8e33e1c523b4f4b'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:44.926736 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:44.927020 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:44.927293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:44.946311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] MFA Rules not processed for user `ffcf7b8ddd454d9fa3b32f86906cedd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:44.947808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:44.966801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1307/2628] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:32:44 2022] DELETE /identity/v3/users/2ee0a86346ff4257a8e33e1c523b4f4b => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:44.989476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:45.005333 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1322/2629] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:32:45.279936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.281405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.350747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90243-633b-410a-9653-da7ce2888b1e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.354437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.355646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.355928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.356203 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.358109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.369315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.371157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023abe7-bb9f-4146-bbc7-1518e7e267a1 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.380621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1308/2630] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:32:45.427024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.428625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.456331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e8437-b8ab-4d0a-80da-f32e6b580e34 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.518633 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.519189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.519463 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.519762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] PATH_INFO: `/v3/projects/acda4529648b41ae94fabf0bba2538be` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.520895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acda4529648b41ae94fabf0bba2538be)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.522790 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.662742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acda4529648b41ae94fabf0bba2538be', 'request_id': 'req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:45.780019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acda4529648b41ae94fabf0bba2538be', 'request_id': 'req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:45.787993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acda4529648b41ae94fabf0bba2538be', 'request_id': 'req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:45.802177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] The token cache is being invalidate because project acda4529648b41ae94fabf0bba2538be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:32:45.802468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:45.803119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-4d72c155-bdc6-4f33-b5e9-6f3aa7cbaad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:32:45.804854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1323/2631] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:32:45 2022] DELETE /identity/v3/projects/acda4529648b41ae94fabf0bba2538be => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:45.810183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.810661 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.810772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.832468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:45.833713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.899088 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1309/2632] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:45.907353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.907657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.907988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.928111 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:45.929331 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:45.938825 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1324/2633] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:45.942983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.954243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784339f9-eeda-42a9-9636-2f0a99365872 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.959625 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.960259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.960534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.961535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.962183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:32:45.962513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.964246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:45.976417 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1310/2634] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:32:45.980854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:45.990322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd28b7-aafc-4e37-9ee6-be3c720845b6 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:45.995000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:45.996093 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:45.996457 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:45.996767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:45.997421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:45.999359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.134690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1325/2635] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:45 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:46.139759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.149672 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529689d-9e1e-4118-a3d5-1d52d639bc0d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.181363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:46.206939 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.207546 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.207853 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.208156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.208811 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.210722 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.297431 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1311/2636] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:46 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:46.302421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.313700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caaafb-ba94-480d-92ba-11676daf7525 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.318435 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.319040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.319318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.319584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.320229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.322122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.332406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1326/2637] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:46.336441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.347177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a3574-739e-4b5a-9ad5-6edb077c5d8a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.351872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.352468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.352787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.353063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.353708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.355784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf824f0-d9eb-44ed-8051-d07ed68cef2f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.366053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1312/2638] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:46.370187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.381685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862f10b-f1bc-4825-be5c-3fe63e510cf8 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.386527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.387172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.387560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.387906 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] PATH_INFO: `/v3/projects/b7f7a8ec58894be78ab4201033844206/users/26493f3dad434e8190877584a12132cf/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.406047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f7a8ec58894be78ab4201033844206, user_id=26493f3dad434e8190877584a12132cf, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.408717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.535735 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1327/2639] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:46 2022] PUT /identity/v3/projects/b7f7a8ec58894be78ab4201033844206/users/26493f3dad434e8190877584a12132cf/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:46.547001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.547288 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.547562 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.575639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] MFA Rules not processed for user `26493f3dad434e8190877584a12132cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:46.576881 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:46.643064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1313/2640] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:46.647431 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.667094 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:46.712577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95d2a-3d0c-4052-84ec-0d30ac6370b0 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.719297 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.719919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.720206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.720482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.721210 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.723189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:46.841130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1328/2641] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:46 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:46.845384 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:46.857138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d03986-266d-41b4-89eb-2fcf099d951e None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:46.916564 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:46.917830 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:46.918594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:46.918735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:46.919286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:46.921571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.039905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1314/2642] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:46 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:47.044281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.054668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f9da9-5e74-44e8-9e29-ae9935c42290 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.059282 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.059884 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.060165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.060439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.061121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.062711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.074090 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1329/2643] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:47.078158 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.088011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3da5-ed99-42bf-a161-1766ff9027d6 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.093335 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.093899 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.094178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.094452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.095092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.096708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae083dc-8222-46ae-9599-488f58d9ca60 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.107003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1315/2644] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:47.110956 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.120855 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb594d49-8275-4fca-a8f4-40cd115c045c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.125330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.125966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.126243 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.126512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] PATH_INFO: `/v3/projects/b18926df9a534324854ed12919aed648/users/cc5503a3b3c048da9699e7620e822702/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.142077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18926df9a534324854ed12919aed648, user_id=cc5503a3b3c048da9699e7620e822702, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.144775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.330819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1330/2645] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:47 2022] PUT /identity/v3/projects/b18926df9a534324854ed12919aed648/users/cc5503a3b3c048da9699e7620e822702/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:47.342804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.343098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.343378 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.370983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] MFA Rules not processed for user `cc5503a3b3c048da9699e7620e822702`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:47.372201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:47.382379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:47.440461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1316/2646] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:47.444988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.508258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bc520-040f-4aa8-95a0-65d13e64ec73 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.516522 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.517172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.517452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.517726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.518415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.520338 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.670632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1331/2647] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:32:47 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:47.674662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.685629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4d02b-57dc-4adf-83b2-d83a554bb652 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.743222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.743832 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.744110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.744383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.745081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.747015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.891865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1317/2648] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:32:47 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:32:47.895713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.906980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f3b8b-1223-4200-9742-44bfc20a829d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.911595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.912186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.912472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.912771 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.913419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.915042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:47.925493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1332/2649] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:47.929741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:47.940012 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c9362-360c-45d2-8349-bcf1cf71ee9f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:47.944810 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:47.945798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:47.946082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:47.946408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] PATH_INFO: `/v3/projects/c02a9d735cae4cad968305e5869ec4f3/users/f4e3221ac71f459abbc53d0977bc1306/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:47.962606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02a9d735cae4cad968305e5869ec4f3, user_id=f4e3221ac71f459abbc53d0977bc1306, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:47.965916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.070020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1318/2650] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:47 2022] PUT /identity/v3/projects/c02a9d735cae4cad968305e5869ec4f3/users/f4e3221ac71f459abbc53d0977bc1306/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:48.078714 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.088577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b7824-d3db-414b-b1ac-612eb5da5e96 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.127957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:48.146459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.147095 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.147377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.147650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.148322 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.149952 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ff0831-a238-46e9-9a1b-5a6d4b4bae10 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.160709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1333/2651] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:32:48.164836 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.176074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a96906-7c0d-43cc-91d8-553d70730641 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.180764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.181419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.181701 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.181979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] PATH_INFO: `/v3/projects/c02a9d735cae4cad968305e5869ec4f3/users/f4e3221ac71f459abbc53d0977bc1306/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.184180 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02a9d735cae4cad968305e5869ec4f3, user_id=f4e3221ac71f459abbc53d0977bc1306, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.186854 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.197439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1319/2652] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:32:48 2022] PUT /identity/v3/projects/c02a9d735cae4cad968305e5869ec4f3/users/f4e3221ac71f459abbc53d0977bc1306/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:32:48.207787 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.218029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.218029 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.237453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] MFA Rules not processed for user `f4e3221ac71f459abbc53d0977bc1306`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:32:48.238661 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:48.305579 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1334/2653] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:32:48.316856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:48.318579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:48.380988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1033a6-b4e3-42f3-ae3e-c389965ffc1b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:48.384621 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:48.385252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:48.385534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:48.385842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:48.394872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:48.396294 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:48.406606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1320/2654] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:50.026176 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:50.037209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249a4b-15e9-4fbc-9b0b-644437e61c73 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:50.040772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:50.041364 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:50.041641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:50.041912 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:50.099284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:50.100991 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:50.110585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1335/2655] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:55.305531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:55.310273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:55.324517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0d3dc-6c1c-4421-af3d-96e6a8c24180 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:55.335234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:55.335234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:55.335435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:55.335829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:55.341595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:55.397436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:55.398894 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:55.408746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1321/2656] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:32:59.222284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:32:59.223727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.281110 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69486b-60fd-4ef8-b927-068840c356ad service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:32:59.284807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:32:59.285409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:32:59.285687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:32:59.285959 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:32:59.287730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:32:59.300685 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:32:59.353086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:32:59.354520 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74d9d2-7558-45f6-9945-0d6c14b50162 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:32:59.364037 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1336/2657] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:00.436554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.449015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f0c61-94fa-4af3-8e22-2db440aa6ef8 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:00.454007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:00.454967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.455243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.455518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] PATH_INFO: `/v3/users/26493f3dad434e8190877584a12132cf` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.461299 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26493f3dad434e8190877584a12132cf)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:00.468873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:00.726889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26493f3dad434e8190877584a12132cf'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.735254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26493f3dad434e8190877584a12132cf'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.756785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26493f3dad434e8190877584a12132cf'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.757547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26493f3dad434e8190877584a12132cf'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.758154 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26493f3dad434e8190877584a12132cf'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:00.851215 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1322/2658] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:00 2022] DELETE /identity/v3/users/26493f3dad434e8190877584a12132cf => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:00.855047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:00.856152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:00.856152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:00.877039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:00.878368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:00.951138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1337/2659] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:00.960246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:00.961797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:01.036764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159189-573e-4386-bb8a-053bd07a7418 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.040916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.041607 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.041996 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.042328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.052494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.054036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.064529 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1323/2660] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:01.583173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.584830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:01.606610 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e057f52-40bb-48ee-92a4-96c61b67a382 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.617657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.617657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.617657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.617657 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] PATH_INFO: `/v3/projects/b7f7a8ec58894be78ab4201033844206` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.618539 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f7a8ec58894be78ab4201033844206)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.619819 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:01.753512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f7a8ec58894be78ab4201033844206', 'request_id': 'req-f36d2f4f-71b4-424e-ba55-fb536e5b7021'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:01.818680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f7a8ec58894be78ab4201033844206', 'request_id': 'req-f36d2f4f-71b4-424e-ba55-fb536e5b7021'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:01.826429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f7a8ec58894be78ab4201033844206', 'request_id': 'req-f36d2f4f-71b4-424e-ba55-fb536e5b7021'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:01.838294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] The token cache is being invalidate because project b7f7a8ec58894be78ab4201033844206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:01.838584 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:01.840924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:01.841704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1338/2661] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:01 2022] DELETE /identity/v3/projects/b7f7a8ec58894be78ab4201033844206 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:01.845679 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:01.863317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b435f3-d60f-4926-888a-448022e801a8 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:01.888233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:01.945348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:01.946907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:01.947933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:01.950980 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] PATH_INFO: `/v3/users/cc5503a3b3c048da9699e7620e822702` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:01.952889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc5503a3b3c048da9699e7620e822702)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:01.956850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:02.171313 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc5503a3b3c048da9699e7620e822702'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:02.178291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc5503a3b3c048da9699e7620e822702'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:02.191838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc5503a3b3c048da9699e7620e822702'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:02.192547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc5503a3b3c048da9699e7620e822702'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:02.193274 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc5503a3b3c048da9699e7620e822702'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:02.278581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1324/2662] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:01 2022] DELETE /identity/v3/users/cc5503a3b3c048da9699e7620e822702 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:02.858074 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:02.859668 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:02.887357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d2f4f-71b4-424e-ba55-fb536e5b7021 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:02.924222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:02.957625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:02.958261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:02.958551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:02.959319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] PATH_INFO: `/v3/projects/b18926df9a534324854ed12919aed648` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:02.960475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18926df9a534324854ed12919aed648)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:02.962445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:03.130448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18926df9a534324854ed12919aed648', 'request_id': 'req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.204353 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18926df9a534324854ed12919aed648', 'request_id': 'req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.210737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18926df9a534324854ed12919aed648', 'request_id': 'req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.222628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] The token cache is being invalidate because project b18926df9a534324854ed12919aed648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:03.222918 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.223680 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.225350 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1339/2663] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:02 2022] DELETE /identity/v3/projects/b18926df9a534324854ed12919aed648 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:03.229329 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:03.230882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:03.244096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3c287-ae70-4317-b781-2eadc3846bba admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:03.303043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:03.303612 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:03.303926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:03.304156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] PATH_INFO: `/v3/users/f4e3221ac71f459abbc53d0977bc1306` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:03.305304 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e3221ac71f459abbc53d0977bc1306)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:03.307185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:03.551186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:03.556276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e3221ac71f459abbc53d0977bc1306'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.564094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e3221ac71f459abbc53d0977bc1306'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.581183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e3221ac71f459abbc53d0977bc1306'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.581183 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e3221ac71f459abbc53d0977bc1306'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.585937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:03.587573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:03.588994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e3221ac71f459abbc53d0977bc1306'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.605740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b528-66d0-4f47-8f68-fd868ede7a9c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:03.610371 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:03.610996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:03.613916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:03.613916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] PATH_INFO: `/v3/users/d451fb2f26ce4f50b010fb2a387c4bf3` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:03.613916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d451fb2f26ce4f50b010fb2a387c4bf3)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:03.614563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:03.730814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1325/2664] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:03 2022] DELETE /identity/v3/users/f4e3221ac71f459abbc53d0977bc1306 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:03.877895 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd451fb2f26ce4f50b010fb2a387c4bf3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.884726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd451fb2f26ce4f50b010fb2a387c4bf3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.900618 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd451fb2f26ce4f50b010fb2a387c4bf3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.901432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd451fb2f26ce4f50b010fb2a387c4bf3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.902586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd451fb2f26ce4f50b010fb2a387c4bf3'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:03.974577 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1340/2665] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:03 2022] DELETE /identity/v3/users/d451fb2f26ce4f50b010fb2a387c4bf3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:04.230403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.230403 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.266332 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.267926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.298315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170357b-509b-4f93-8f64-bd4824bc3f84 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.315040 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:04.341190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa03b-445b-4e13-be6a-e68e579a8d96 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.346160 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.346792 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.347078 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.347376 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] PATH_INFO: `/v3/projects/7caa43bf43ad4e84b4b648e9ab33bd7f` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.353975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7caa43bf43ad4e84b4b648e9ab33bd7f)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.353975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:04.359948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.360525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.360943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.361225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] PATH_INFO: `/v3/projects/c02a9d735cae4cad968305e5869ec4f3` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.362381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02a9d735cae4cad968305e5869ec4f3)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.364300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:04.495230 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'request_id': 'req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.531459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02a9d735cae4cad968305e5869ec4f3', 'request_id': 'req-fc5c18e7-7060-49c1-ae0d-678c0a898865'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.576117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'request_id': 'req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.582596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7caa43bf43ad4e84b4b648e9ab33bd7f', 'request_id': 'req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.595551 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] The token cache is being invalidate because project 7caa43bf43ad4e84b4b648e9ab33bd7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:04.595851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.596499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-cbcf084a-6c61-4fd3-ade3-bbafbd9dfb0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.598139 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1341/2666] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:04 2022] DELETE /identity/v3/projects/7caa43bf43ad4e84b4b648e9ab33bd7f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:04.602958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbf24-5fa5-4e0f-8c4d-ddf1214a583a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.603239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbf24-5fa5-4e0f-8c4d-ddf1214a583a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.603512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbf24-5fa5-4e0f-8c4d-ddf1214a583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.625450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9d2bbf24-5fa5-4e0f-8c4d-ddf1214a583a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:04.626572 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9d2bbf24-5fa5-4e0f-8c4d-ddf1214a583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.684987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1342/2667] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:04.692858 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.693145 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.693446 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.712279 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:04.712646 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02a9d735cae4cad968305e5869ec4f3', 'request_id': 'req-fc5c18e7-7060-49c1-ae0d-678c0a898865'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.713528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.720315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02a9d735cae4cad968305e5869ec4f3', 'request_id': 'req-fc5c18e7-7060-49c1-ae0d-678c0a898865'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.722704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1343/2668] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:04.727128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.730367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] The token cache is being invalidate because project c02a9d735cae4cad968305e5869ec4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:04.730657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.731908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-fc5c18e7-7060-49c1-ae0d-678c0a898865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:04.733490 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1326/2669] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:04 2022] DELETE /identity/v3/projects/c02a9d735cae4cad968305e5869ec4f3 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:04.737031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6348f1e-c38d-473c-b32a-58e821de3a40 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.740444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2586-9562-4a4f-af11-73c955b4359a None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.740753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2586-9562-4a4f-af11-73c955b4359a None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.741027 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2586-9562-4a4f-af11-73c955b4359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.743713 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.744533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.744850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.745164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.745843 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:04.746135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.747751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:04.760298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1344/2670] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:33:04.762454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-546f2586-9562-4a4f-af11-73c955b4359a None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:04.764043 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-546f2586-9562-4a4f-af11-73c955b4359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.765406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.767374 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.776807 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1327/2671] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:04.781741 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5cb44-4346-4a07-b15d-dbafc51e4030 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.798283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852ac9d-3080-49c8-a43c-9087ff36c512 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.798283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852ac9d-3080-49c8-a43c-9087ff36c512 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.798283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852ac9d-3080-49c8-a43c-9087ff36c512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.799761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.799761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.799761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.799761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.800700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.802568 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:04.826991 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-c852ac9d-3080-49c8-a43c-9087ff36c512 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:04.828202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c852ac9d-3080-49c8-a43c-9087ff36c512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.836156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1328/2672] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:04.837338 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.837620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.837890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.859423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] MFA Rules not processed for user `ffcf7b8ddd454d9fa3b32f86906cedd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:04.860610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:04.906573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1345/2673] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:04.906573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.919787 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1329/2674] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:04.924635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:04.935156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab583-8e0f-4e39-8977-3654a56e28fd None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.980377 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da5cc8-79f3-44d0-a4f5-3e8f46e23142 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:04.986806 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.987418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.987709 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.987965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.988920 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:04.989229 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.990820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:04.993694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:04.994380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:04.994734 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:04.995084 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:04.995781 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:04.997889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.004988 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1346/2675] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:05.009187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.020759 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ac90-6238-4339-9d20-2a2cc323de9f admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.025439 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.026034 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.026312 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.026581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.027197 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.030358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.036207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:05.122790 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1330/2676] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:04 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:05.127209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.139106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436aad-0666-4121-a09f-f946ef9e0acf admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.145847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.146472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.146755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.148983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.148983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.149488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.154590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1347/2677] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:05.159086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.168418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1331/2678] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:05.170032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.180578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bde417-80fe-426b-9dc3-7834e7691035 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.229271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b4a58-e4e8-4b0b-85a9-425e05a0857e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.234940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.235571 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.235854 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.236133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.236907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.238782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.248174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.249500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.249972 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.250910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.251035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.252575 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.264207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1332/2679] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:05.267872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.279221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a8fe8-8cbd-442d-aa2d-ea32a795ec2a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.284190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.284871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.285152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.285425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] PATH_INFO: `/v3/projects/962280636320471080abda11086f1e9e/users/78e6c594dd7f4cd8b20c1361c1eadcc9/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.300315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=962280636320471080abda11086f1e9e, user_id=78e6c594dd7f4cd8b20c1361c1eadcc9, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.303106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b88653-bd02-4410-98bf-6f6e0d3593a0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.353528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1348/2680] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:05 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:05.354640 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.364763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714014-7662-49f0-94b2-8c559628fe3b admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.369518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.370118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.373121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.373121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.373121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.373676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.383849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1349/2681] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:05.388572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1333/2682] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:05 2022] PUT /identity/v3/projects/962280636320471080abda11086f1e9e/users/78e6c594dd7f4cd8b20c1361c1eadcc9/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:05.388896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.399148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.399648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.400518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.402399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e7794-f262-4a0d-a080-8290145a2a9e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.432874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] MFA Rules not processed for user `78e6c594dd7f4cd8b20c1361c1eadcc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:05.435375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:05.469821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.470511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.470798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.471103 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.471760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.473517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.485117 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1350/2683] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:05.489812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.500992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c80ca-a5ba-4b4b-be17-a56a6b26159e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.505757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.506418 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.506727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.507002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] PATH_INFO: `/v3/projects/2680aa11074448e6b2adced27b844f31/users/454026c435ff4b209453ac53e5b77359/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.507694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1334/2684] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:05.519763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.519968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:05.522517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:05.526707 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2680aa11074448e6b2adced27b844f31, user_id=454026c435ff4b209453ac53e5b77359, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.530634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152970ee-0aa9-4b21-abf7-3dca5d91df7d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.607696 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3959c5-5997-4434-8a57-c4b88ccc2ca2 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.613376 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.614028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.614491 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.614618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.622622 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.624194 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.633206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1351/2685] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:05 2022] PUT /identity/v3/projects/2680aa11074448e6b2adced27b844f31/users/454026c435ff4b209453ac53e5b77359/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:05.634504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1335/2686] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:05.644874 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.645157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.645429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.675514 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] MFA Rules not processed for user `454026c435ff4b209453ac53e5b77359`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:05.676820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:05.743130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:05.747694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1352/2687] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:05.759517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:05.761045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:05.834796 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfeb8cf-b8b2-4afb-b27d-fbedd12e2f20 service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:05.861778 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:05.888557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:05.888737 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:05.889059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:05.889291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:05.898845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:05.900318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03026f5b-f7c3-419b-baa2-fef67efdac83 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:05.911507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1336/2688] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:05.918555 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:16.111798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:16.178215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137e1dc-db87-4925-8685-5a18a918d02e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:16.182934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:16.183581 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:16.183909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:16.184187 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] PATH_INFO: `/v3/users/78e6c594dd7f4cd8b20c1361c1eadcc9` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:16.185559 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e6c594dd7f4cd8b20c1361c1eadcc9)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:16.188302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:16.439660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e6c594dd7f4cd8b20c1361c1eadcc9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:16.448114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e6c594dd7f4cd8b20c1361c1eadcc9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:16.469215 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e6c594dd7f4cd8b20c1361c1eadcc9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:16.473389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e6c594dd7f4cd8b20c1361c1eadcc9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:16.473389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e6c594dd7f4cd8b20c1361c1eadcc9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:16.475044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:16.613519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1353/2689] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:16 2022] DELETE /identity/v3/users/78e6c594dd7f4cd8b20c1361c1eadcc9 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:16.618128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:16.618613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:16.618719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:16.637903 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:16.639087 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:16.700078 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1337/2690] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:16.710143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:16.711535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:16.774705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb99e42-f19b-4c61-8251-4fe18d450e4d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:16.778309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:16.783827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:16.783827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:16.783827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:16.789834 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:16.791293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b604f-260c-4f08-b4af-c3e2cf19b638 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:16.801777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1354/2691] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:17.276816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.276816 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.295054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd152e08-c64b-466c-815c-4b434ba09157 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.301343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.301989 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.302263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.302534 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] PATH_INFO: `/v3/projects/962280636320471080abda11086f1e9e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.303673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=962280636320471080abda11086f1e9e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.305825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:17.440873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962280636320471080abda11086f1e9e', 'request_id': 'req-e076dc43-0a08-43ed-b665-5c6b5154ec47'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.517015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962280636320471080abda11086f1e9e', 'request_id': 'req-e076dc43-0a08-43ed-b665-5c6b5154ec47'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.523109 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962280636320471080abda11086f1e9e', 'request_id': 'req-e076dc43-0a08-43ed-b665-5c6b5154ec47'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.534478 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] The token cache is being invalidate because project 962280636320471080abda11086f1e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:17.534760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.535377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e076dc43-0a08-43ed-b665-5c6b5154ec47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:17.536890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1338/2692] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:17 2022] DELETE /identity/v3/projects/962280636320471080abda11086f1e9e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:17.541785 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.542273 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.542372 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.563271 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:17.564473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.630495 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1355/2693] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:17.643994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.644504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.644719 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.697748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:17.729156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:17.730381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:17.780890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1339/2694] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:17.785251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.839600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00a82f-95d3-4b32-9a28-9c5835f6f6c8 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.844865 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.845586 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.845862 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.846137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.846793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:17.847096 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.848847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:17.862083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1356/2695] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:17.866623 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:17.876468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd1357-cbef-4b1b-acca-401414cf97e7 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:17.881851 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:17.882459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:17.882788 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:17.883063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:17.883746 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:17.885862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.006161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1340/2696] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:17 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:18.010391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.020387 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b0c4-1918-4fdd-9547-690f92836bb5 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.065262 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:18.078267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.078869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.079143 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.079415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.080140 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.082077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.176203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1357/2697] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:18 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:18.180351 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.191470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a9f74-69e6-4f24-9ce4-b5042f45df1a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.196074 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.196765 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.197046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.197563 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.198212 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.199783 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.210611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1341/2698] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:18.214448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.224342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccd096-d42c-4bef-bceb-e1771d3cc8d9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.228914 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.229573 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.229849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.230118 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.230757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.232347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce00524-5dcb-4fdd-9e42-8a9b421c585f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.244359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1358/2699] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:18.248630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.259172 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74303cb6-d9de-454d-8acb-3998cdfea72d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.263723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.264367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.265588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.265588 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] PATH_INFO: `/v3/projects/17403fc1a5344fc4a1466c1817cda6b4/users/07c2af83913d43f19385eab768da872a/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.279868 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17403fc1a5344fc4a1466c1817cda6b4, user_id=07c2af83913d43f19385eab768da872a, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.282760 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.376196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1342/2700] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:18 2022] PUT /identity/v3/projects/17403fc1a5344fc4a1466c1817cda6b4/users/07c2af83913d43f19385eab768da872a/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:18.386864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.390241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.390241 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.416925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] MFA Rules not processed for user `07c2af83913d43f19385eab768da872a`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:18.418429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:18.485576 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1359/2701] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:18.490108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.553362 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82b66f-7fc2-4b10-ada1-6d263c661ad3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.557984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.558585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.558859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.559128 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.559753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.562913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.683573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1343/2702] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:18 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:18.684747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.686563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:18.699427 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.701266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814582b9-d2c2-47d8-bf1a-b55bad7327da None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.728654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:18.759653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.760388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.760602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.760919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] PATH_INFO: `/v3/users/454026c435ff4b209453ac53e5b77359` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.762201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454026c435ff4b209453ac53e5b77359)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.764099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58f39-0eef-4196-bf4b-60bd1f951874 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.764224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.769761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.770465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.770861 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.771131 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.771762 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.773770 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.858823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1344/2703] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:18 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:18.862755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.872399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56481-0c53-4a52-b6c7-3a751c27fb35 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.878091 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.878673 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.878951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.879222 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.879855 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.881470 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.892326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1345/2704] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:18.896408 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:18.906105 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69344c2a-d107-4681-bfda-fef28721c7c5 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:18.910631 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:18.911257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:18.911533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:18.911804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] PATH_INFO: `/v3/projects/90be484f2027476785f27b82488f4336/users/53d6148901514372a031dddfc6f09523/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:18.926278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90be484f2027476785f27b82488f4336, user_id=53d6148901514372a031dddfc6f09523, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:18.928904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:18.966935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454026c435ff4b209453ac53e5b77359'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.975924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454026c435ff4b209453ac53e5b77359'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.977098 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:18.989358 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454026c435ff4b209453ac53e5b77359'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.990150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454026c435ff4b209453ac53e5b77359'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:18.990776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-f696f9b2-4cd0-42ae-b12e-2c92d9358054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454026c435ff4b209453ac53e5b77359'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:19.016508 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1346/2705] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:18 2022] PUT /identity/v3/projects/90be484f2027476785f27b82488f4336/users/53d6148901514372a031dddfc6f09523/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:19.021404 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.022124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.087834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcdfbe-d3fe-4852-9517-0e5049ae0c30 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.092859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.093620 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.093904 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.094181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.094825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.096413 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.106265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1347/2706] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:19.113082 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.119651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b105-fdb0-4076-aeee-c0c955045811 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.124711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.125324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.125601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.125874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] PATH_INFO: `/v3/projects/90be484f2027476785f27b82488f4336/users/53d6148901514372a031dddfc6f09523/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.127794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90be484f2027476785f27b82488f4336, user_id=53d6148901514372a031dddfc6f09523, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.130412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a14239-4d88-43a0-8b6b-98c1df90c606 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.138756 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1360/2707] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:18 2022] DELETE /identity/v3/users/454026c435ff4b209453ac53e5b77359 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:19.142281 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1348/2708] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:19 2022] PUT /identity/v3/projects/90be484f2027476785f27b82488f4336/users/53d6148901514372a031dddfc6f09523/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:19.143474 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.143755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.144026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.152266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.152628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.153043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.165533 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:19.166752 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.182528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] MFA Rules not processed for user `53d6148901514372a031dddfc6f09523`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:19.183742 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.229356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1349/2709] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:19.239775 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.241182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.243915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:19.259137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1361/2710] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:19.271885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.273264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.315911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2721e8-0d7f-4e81-9231-8933becf0bc4 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.319632 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.320291 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.320571 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.320878 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.330493 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.332026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.339797 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d31e5-1a13-477f-acfa-a17853bdae08 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.341707 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1350/2711] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:19.345231 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.345850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.346127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.346399 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.355764 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.357285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.368580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1362/2712] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:19.475929 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.486265 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e6e55-f4d9-479f-b7e7-128b22e05fa5 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.490014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.490600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.490879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.491148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.492934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.554301 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.555730 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5b727-a142-4c99-8716-c206d99d6d64 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:19.565741 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1351/2713] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:19.833795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:19.844375 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:19.863533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ee46a-79b2-4826-b83f-894803ff1348 service cinder] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:19.869329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:19.870201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:19.871061 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:19.871347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] PATH_INFO: `/v3/projects/2680aa11074448e6b2adced27b844f31` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:19.872528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2680aa11074448e6b2adced27b844f31)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:19.875442 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.011127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2680aa11074448e6b2adced27b844f31', 'request_id': 'req-26155662-3360-4075-aa97-5e5ca9f5369f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:20.067313 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2680aa11074448e6b2adced27b844f31', 'request_id': 'req-26155662-3360-4075-aa97-5e5ca9f5369f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:20.069542 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:20.076142 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2680aa11074448e6b2adced27b844f31', 'request_id': 'req-26155662-3360-4075-aa97-5e5ca9f5369f'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:20.091122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] The token cache is being invalidate because project 2680aa11074448e6b2adced27b844f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:20.091424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:20.092203 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-26155662-3360-4075-aa97-5e5ca9f5369f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:20.094799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1363/2714] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:19 2022] DELETE /identity/v3/projects/2680aa11074448e6b2adced27b844f31 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:20.099755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.100233 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.100345 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.122499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:20.127402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:20.190231 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1352/2715] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:20.197638 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.198010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.198359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.226026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:20.228043 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:20.238000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1364/2716] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:20.242201 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.252797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6062711-d135-4d17-8694-f29daac72ea6 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.257488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.258080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.258359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.258634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.259359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:20.259665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.261286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.273911 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1353/2717] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:20.278180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.287465 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f1379-889c-47ea-9ea1-7ad3221c77a0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.292254 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.292881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.293157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.293420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.294033 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.296434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.424075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1365/2718] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:20.429119 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.443420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b602-0e67-45ba-9394-73e3d340a06a admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.461032 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:20.511151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.511758 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.512037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.512308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.513786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.516501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.648581 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1354/2719] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:20.652560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.663330 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa4c2-11e4-406e-bf50-1185bd532739 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.667820 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.668424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.668726 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.669002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.669642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.671980 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.681881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1366/2720] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:20.686271 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.696444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391eac2-e965-4c26-87ed-6712efed4ef0 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.701471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.702766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.703113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.703393 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.704083 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.705694 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102a553-4bf6-400c-a034-1823fe2c5baf admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.716017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1355/2721] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:20.721026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.731065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037520b4-f9f4-43b3-90d2-c35ee21da5f9 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:20.736236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:20.737009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.737310 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.737589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] PATH_INFO: `/v3/projects/38628e1a0d9b41dc9feb9bb102641ec2/users/3df08ebbcddd47caa9c68c7901534b10/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.753209 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38628e1a0d9b41dc9feb9bb102641ec2, user_id=3df08ebbcddd47caa9c68c7901534b10, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:20.756007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:20.858727 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1367/2722] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:20 2022] PUT /identity/v3/projects/38628e1a0d9b41dc9feb9bb102641ec2/users/3df08ebbcddd47caa9c68c7901534b10/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:20.869955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:20.870237 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:20.870507 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:20.900296 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] MFA Rules not processed for user `3df08ebbcddd47caa9c68c7901534b10`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:20.901669 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:20.971381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1356/2723] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:20.983751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:20.986010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:21.052976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6ff12-9791-479f-b12a-a7e061cbd34a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:21.056911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:21.057540 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:21.057815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:21.058100 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:21.067518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:21.068972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee842e-e33d-4210-8e9a-ec08d36202b3 service glance] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:21.078614 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1368/2724] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:22.916072 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:22.917664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:22.987279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e607-b332-4b6c-9108-d5858c6a21d0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:22.992156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:22.992866 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:22.993153 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:22.993430 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] PATH_INFO: `/v3/users/07c2af83913d43f19385eab768da872a` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:22.994616 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c2af83913d43f19385eab768da872a)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:22.996912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:23.090856 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:23.212924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c2af83913d43f19385eab768da872a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:23.220841 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c2af83913d43f19385eab768da872a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:23.235877 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c2af83913d43f19385eab768da872a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:23.236706 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c2af83913d43f19385eab768da872a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:23.237324 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c2af83913d43f19385eab768da872a'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:23.328503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1357/2725] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:22 2022] DELETE /identity/v3/users/07c2af83913d43f19385eab768da872a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:23.332993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:23.333475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:23.333591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:23.353007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:23.354534 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:23.419453 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1369/2726] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:23.430356 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:23.431804 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:23.497506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86956b53-08f4-4c39-8d70-394246718b73 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:23.501238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:23.501849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:23.502129 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:23.502399 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:23.511879 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:23.513341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:23.522937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1358/2727] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:23.982525 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:23.984519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:23.998249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f18a5-14c3-419a-9aab-a5a2082fe690 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:24.003444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:24.004125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:24.004580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:24.004723 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] PATH_INFO: `/v3/projects/17403fc1a5344fc4a1466c1817cda6b4` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:24.005833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17403fc1a5344fc4a1466c1817cda6b4)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:24.007760 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:24.032081 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:24.128414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17403fc1a5344fc4a1466c1817cda6b4', 'request_id': 'req-ba8956b1-5a3a-4c1d-8079-7ac125e52435'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.207021 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17403fc1a5344fc4a1466c1817cda6b4', 'request_id': 'req-ba8956b1-5a3a-4c1d-8079-7ac125e52435'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.213097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17403fc1a5344fc4a1466c1817cda6b4', 'request_id': 'req-ba8956b1-5a3a-4c1d-8079-7ac125e52435'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.224996 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] The token cache is being invalidate because project 17403fc1a5344fc4a1466c1817cda6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:24.225413 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.226121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.227682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1370/2728] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:23 2022] DELETE /identity/v3/projects/17403fc1a5344fc4a1466c1817cda6b4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:24.231967 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:24.242466 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8efc05-f52e-44e0-8c2e-d189cefd5344 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:24.304238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:24.304888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:24.305165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:24.305436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] PATH_INFO: `/v3/users/53d6148901514372a031dddfc6f09523` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:24.306541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d6148901514372a031dddfc6f09523)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:24.308447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:24.487789 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d6148901514372a031dddfc6f09523'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.489504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:24.495901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d6148901514372a031dddfc6f09523'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.508948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d6148901514372a031dddfc6f09523'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.509692 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d6148901514372a031dddfc6f09523'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.510420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ea84f736-47b9-45b5-9edd-7251499f5d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d6148901514372a031dddfc6f09523'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:24.627515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1359/2729] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:24 2022] DELETE /identity/v3/users/53d6148901514372a031dddfc6f09523 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:25.211847 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:25.213496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:25.227010 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8956b1-5a3a-4c1d-8079-7ac125e52435 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:25.295924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:25.296563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.298135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.298427 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] PATH_INFO: `/v3/projects/90be484f2027476785f27b82488f4336` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.299666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90be484f2027476785f27b82488f4336)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:25.301815 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:25.428267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90be484f2027476785f27b82488f4336', 'request_id': 'req-3e7c633c-9ae4-4a1f-8d35-9810688c4310'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:25.501591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90be484f2027476785f27b82488f4336', 'request_id': 'req-3e7c633c-9ae4-4a1f-8d35-9810688c4310'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:25.509477 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90be484f2027476785f27b82488f4336', 'request_id': 'req-3e7c633c-9ae4-4a1f-8d35-9810688c4310'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:25.521295 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] The token cache is being invalidate because project 90be484f2027476785f27b82488f4336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:25.521607 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:25.522265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-3e7c633c-9ae4-4a1f-8d35-9810688c4310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:25.523748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1371/2730] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:25 2022] DELETE /identity/v3/projects/90be484f2027476785f27b82488f4336 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:25.529063 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.531708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.532031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.554858 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:25.556080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:25.628266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1360/2731] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:25.637188 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.637474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.637748 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.659247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:25.660506 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:25.668629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1372/2732] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:25.674041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:25.684198 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc3dd-a5d1-41c0-b64a-f586b3521a13 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:25.689193 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:25.689819 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.690099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.690373 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.691068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:25.691440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:25.693041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:25.706573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1361/2733] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:25.710583 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:25.726650 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00921d9-a694-44af-9e4a-8b267761a21d None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:25.731574 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:25.732183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.732464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.732772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.733443 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:25.736195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:25.852270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1373/2734] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:25 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:25.856495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:25.867755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5794f-d636-4b68-b248-3d29ba9d8526 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:25.932359 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:25.932876 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:25.933168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:25.933512 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:25.934257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:25.936257 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.023880 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.025624 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:26.040024 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:26.047443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1362/2735] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:25 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:26.053794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.064256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f700e40-a6fc-4b4d-b869-cf13039e0f3f admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.073641 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.074290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.074570 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.074846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.075509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.077243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.089367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1363/2736] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:26.094629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.104489 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40ee87-7fdf-4dcb-93cf-e17e4444a965 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.109415 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.110045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.110330 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.110604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.111267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.112944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.123402 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1364/2737] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:26.127976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.133147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35eb54-c006-4cf9-848c-e376de4d220d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.137896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012d2df-0a6a-4a23-8524-6d10982a2051 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.138777 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.139417 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.139695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.139966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.145833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:26.148747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.148747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.148747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.148747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] PATH_INFO: `/v3/projects/1a685bd59acd4febb48575f75f3ac3c7/users/9afbfe04fa8543caa72501b7ae99a62e/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.154769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.157496 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c68c3d-8296-448f-a8dd-315e8765a20e service placement] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.162961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a685bd59acd4febb48575f75f3ac3c7, user_id=9afbfe04fa8543caa72501b7ae99a62e, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.166981 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.169464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1374/2738] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:26.175831 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:26.230567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1365/2739] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:26 2022] PUT /identity/v3/projects/1a685bd59acd4febb48575f75f3ac3c7/users/9afbfe04fa8543caa72501b7ae99a62e/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:26.241328 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.241801 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.241919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.273664 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] MFA Rules not processed for user `9afbfe04fa8543caa72501b7ae99a62e`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:26.274923 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:26.345628 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1375/2740] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:26.350482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.413463 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276029c-6f54-4b65-b18c-7a4f46024219 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.418279 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.418885 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.419161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.419432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.420064 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.422693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.543137 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1366/2741] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:26 2022] POST /identity/v3/projects => generated 389 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:26.547672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.558464 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c147f-d6d6-47d5-9b72-d4a2baf2a953 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.618236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.618860 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.619137 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.619404 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.620195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.622125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.746992 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:26.756340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1376/2742] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:26 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:26.757567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.772729 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0f486-f4bc-464e-a61c-2e6ff82c30b3 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.777363 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.777968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.778250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.778525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.780471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.780839 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.792425 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1367/2743] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:26.796698 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.811886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a75b92-ba57-449d-8f5b-a1d483c30307 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:26.817410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:26.818135 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:26.818419 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:26.818695 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] PATH_INFO: `/v3/projects/9836337634144f3da0918852d45905ea/users/2f8bf2620af7442db79105f86b56f449/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:26.834151 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9836337634144f3da0918852d45905ea, user_id=2f8bf2620af7442db79105f86b56f449, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:26.836926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:26.935903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1377/2744] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:26 2022] PUT /identity/v3/projects/9836337634144f3da0918852d45905ea/users/2f8bf2620af7442db79105f86b56f449/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:26.940547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:26.950817 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452475dc-59ce-41c9-a331-aa68f89c8896 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:27.012468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:27.013318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.013713 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.013986 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.014634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:27.016254 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399f5f-491a-4442-ac94-a2b26acfac7c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:27.026444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1368/2745] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:27.030475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:27.042682 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91647bc9-44bf-43aa-b370-1a6d6b27bf25 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:27.047761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:27.049180 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.049468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.049747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] PATH_INFO: `/v3/projects/9836337634144f3da0918852d45905ea/users/2f8bf2620af7442db79105f86b56f449/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.051711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9836337634144f3da0918852d45905ea, user_id=2f8bf2620af7442db79105f86b56f449, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:27.054456 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:27.075652 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1378/2746] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:27 2022] PUT /identity/v3/projects/9836337634144f3da0918852d45905ea/users/2f8bf2620af7442db79105f86b56f449/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:27.087139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.087423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.087698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.118232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] MFA Rules not processed for user `2f8bf2620af7442db79105f86b56f449`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:27.119838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:27.175842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:27.205045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1369/2747] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:27.206430 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:27.208493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:27.275263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70533b-9570-4dfd-9a99-13940435b2f0 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:27.279357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:27.280087 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.280502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.280816 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.290629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:27.292059 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5ab39-3d10-43c3-81d9-e9214c33ba19 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:27.302493 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1379/2748] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:27.475004 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:27.539976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0cae1-125c-4b77-ac03-1d39bf932ae5 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:27.544545 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:27.545224 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.545502 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.545771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] PATH_INFO: `/v3/users/9afbfe04fa8543caa72501b7ae99a62e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.546889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afbfe04fa8543caa72501b7ae99a62e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:27.549592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:27.753888 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afbfe04fa8543caa72501b7ae99a62e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:27.761500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afbfe04fa8543caa72501b7ae99a62e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:27.775366 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afbfe04fa8543caa72501b7ae99a62e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:27.776080 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afbfe04fa8543caa72501b7ae99a62e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:27.776711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afbfe04fa8543caa72501b7ae99a62e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:27.872738 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1370/2749] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:27 2022] DELETE /identity/v3/users/9afbfe04fa8543caa72501b7ae99a62e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:27.877982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:27.878265 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:27.878544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:27.897982 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:27.899173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:27.973403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1380/2750] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:27.984283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:27.986098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:28.046120 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be114-2d88-44ca-a5ac-92f3866edc6b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:28.050480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:28.051099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:28.051377 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:28.051653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:28.061516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:28.062956 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:28.072983 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1371/2751] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:28.572023 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:28.573773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:28.591937 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc14ed7-d7fd-40b7-be19-bb9ff1bd127c None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:28.597690 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:28.602030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:28.602030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:28.602030 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] PATH_INFO: `/v3/projects/1a685bd59acd4febb48575f75f3ac3c7` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:28.604194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a685bd59acd4febb48575f75f3ac3c7)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:28.606263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:28.736294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a685bd59acd4febb48575f75f3ac3c7', 'request_id': 'req-296c8611-f456-4f0c-be50-7cad8ec80101'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:28.809560 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a685bd59acd4febb48575f75f3ac3c7', 'request_id': 'req-296c8611-f456-4f0c-be50-7cad8ec80101'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:28.812016 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:28.816710 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a685bd59acd4febb48575f75f3ac3c7', 'request_id': 'req-296c8611-f456-4f0c-be50-7cad8ec80101'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:28.828015 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] The token cache is being invalidate because project 1a685bd59acd4febb48575f75f3ac3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:28.828285 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:28.829948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:28.831233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1381/2752] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:28 2022] DELETE /identity/v3/projects/1a685bd59acd4febb48575f75f3ac3c7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:28.835479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:28.845540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49807c-d4d7-409e-b37b-510ee8513ccb service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:28.855133 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:28.904838 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:28.905469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:28.905761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:28.906059 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] PATH_INFO: `/v3/users/2f8bf2620af7442db79105f86b56f449` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:28.907395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8bf2620af7442db79105f86b56f449)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:28.909388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:29.109243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8bf2620af7442db79105f86b56f449'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:29.116558 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8bf2620af7442db79105f86b56f449'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:29.128930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8bf2620af7442db79105f86b56f449'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:29.305332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8bf2620af7442db79105f86b56f449'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:29.305332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-03fd7c92-5545-4f3f-8ac9-aaf7f64402e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8bf2620af7442db79105f86b56f449'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:29.312871 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1372/2753] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:28 2022] DELETE /identity/v3/users/2f8bf2620af7442db79105f86b56f449 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:29.860069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:29.861737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:29.876424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c8611-f456-4f0c-be50-7cad8ec80101 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:29.938757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:29.939391 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:29.939669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:29.939941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] PATH_INFO: `/v3/projects/9836337634144f3da0918852d45905ea` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:29.941091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9836337634144f3da0918852d45905ea)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:29.943004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.057585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9836337634144f3da0918852d45905ea', 'request_id': 'req-6d89230d-aa09-4d92-82b4-6720c78e770a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.121272 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9836337634144f3da0918852d45905ea', 'request_id': 'req-6d89230d-aa09-4d92-82b4-6720c78e770a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.128985 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9836337634144f3da0918852d45905ea', 'request_id': 'req-6d89230d-aa09-4d92-82b4-6720c78e770a'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.140302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] The token cache is being invalidate because project 9836337634144f3da0918852d45905ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:30.140591 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.141507 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-6d89230d-aa09-4d92-82b4-6720c78e770a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:30.143984 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1382/2754] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:29 2022] DELETE /identity/v3/projects/9836337634144f3da0918852d45905ea => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:30.150036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.152118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.152118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.170782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:30.171998 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.234035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1373/2755] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:30.241934 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.242216 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.242487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.263479 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:30.264731 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.273964 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1383/2756] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:30.278176 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.288379 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d333f31-ff19-4208-a269-0b2058eb54bc None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.292892 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.293480 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.293850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.294028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.294812 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:30.295124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.296848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.309711 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1374/2757] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:30.314830 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.324595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d30d6c-8641-4df5-b34c-ec3cc3083651 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.331837 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.332452 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.332845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.333112 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.333876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.337300 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.433979 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1384/2758] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:30.438227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.449205 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca4373-3e04-4179-91a4-fe0455cb8c6b admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.466454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:30.512849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.513468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.513753 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.514030 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.514663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.516860 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.601675 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1375/2759] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:30.605905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.617369 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d48f6-b1d7-46b9-a7f3-29bccd7b7655 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.622004 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.622601 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.622881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.623153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.623798 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.625662 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.636712 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1385/2760] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 02:33:30.640794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.651035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c72509-5ab1-46b7-b2f9-c2dad0a68a65 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.655780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.656380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.656686 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.657188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.657910 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.660168 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5638aa-46a0-4e08-bedf-a47410445c2c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.670843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1376/2761] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:30.675548 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.686267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905091f-9e4d-4420-94eb-1cf150c506de admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:30.691905 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:30.692562 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.692878 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.693152 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] PATH_INFO: `/v3/projects/9b0ca3fc5cc94408b72516b5b5567152/users/13a4b9f4052647ef8a45ba9fe6c2655d/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.708319 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0ca3fc5cc94408b72516b5b5567152, user_id=13a4b9f4052647ef8a45ba9fe6c2655d, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:30.710998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:30.844641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1386/2762] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:30 2022] PUT /identity/v3/projects/9b0ca3fc5cc94408b72516b5b5567152/users/13a4b9f4052647ef8a45ba9fe6c2655d/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:30.855494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:30.855777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:30.856045 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:30.884560 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] MFA Rules not processed for user `13a4b9f4052647ef8a45ba9fe6c2655d`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:30.885795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:30.953435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1377/2763] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:30.958173 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:30.972069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:31.021130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac1ee3-7a25-4179-92ec-d9aad4e3fb1c admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.025744 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.026347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.026629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.026896 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.027511 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.029433 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.231791 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1387/2764] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:31.235913 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.246930 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a415f-af27-4d6a-ba27-95318e3502fa None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.306190 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.306798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.307077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.307348 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.307979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.309916 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.384611 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1378/2765] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:31.388755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.400148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06518a-5e02-47df-b92a-1dfa458a3664 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.404653 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.405267 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.405541 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.405809 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.406444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.408045 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.419569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1388/2766] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:31.423636 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.434411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df848a-ef6c-4e86-a88d-3615e25f3a87 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.439221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.439859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.440138 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.440476 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] PATH_INFO: `/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb/users/b36c288049fa4034b1f420b17c5bb4f7/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.459388 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=879dcb13f5ec40d68d17921ac8d8fceb, user_id=b36c288049fa4034b1f420b17c5bb4f7, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.462400 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.465381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:31.621845 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1379/2767] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:31 2022] PUT /identity/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb/users/b36c288049fa4034b1f420b17c5bb4f7/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:31.625938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.636890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c32c8-cc99-4fb9-b04c-921aae70ad66 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.694065 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.694641 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.694919 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.695194 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.695845 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.697471 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427511b-4dc5-4e2e-8b64-55436b582c8c admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.708839 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1389/2768] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:31.713113 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.725317 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cedb3-505f-42b2-a3b2-6e39ea8df5ff admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.732840 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.733515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.733805 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.734085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] PATH_INFO: `/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb/users/b36c288049fa4034b1f420b17c5bb4f7/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.736134 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879dcb13f5ec40d68d17921ac8d8fceb, user_id=b36c288049fa4034b1f420b17c5bb4f7, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.738901 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:31.756797 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1380/2769] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:31 2022] PUT /identity/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb/users/b36c288049fa4034b1f420b17c5bb4f7/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:31.761195 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b55a-b1d7-4814-8e1f-895f0f2bc30b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.761469 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b55a-b1d7-4814-8e1f-895f0f2bc30b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.761746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b55a-b1d7-4814-8e1f-895f0f2bc30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.792000 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f0c4b55a-b1d7-4814-8e1f-895f0f2bc30b None None] MFA Rules not processed for user `b36c288049fa4034b1f420b17c5bb4f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:31.793268 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f0c4b55a-b1d7-4814-8e1f-895f0f2bc30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.860774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1390/2770] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:31.872155 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:31.873842 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:31.931764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2c159-0e2d-432c-ab73-9d4b53988f24 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:31.935452 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:31.936048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:31.936326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:31.936602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:31.992499 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:31.994009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:32.003053 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1381/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:32.844351 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:32.844869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:32.845927 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:32.858144 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:32.875097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:32.880329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:32.941128 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1391/2772] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:33:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:33.092931 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:33.094334 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:33.163716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968667e6-1585-49ea-b465-9bb18e1e8bb0 service cinder] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:33.168852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:33.169486 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:33.169763 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:33.170036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:33.178829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:33.180410 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:33.188303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1382/2773] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:33:37.639975 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:37.651038 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64aae-7bb5-45e4-9d6a-5a16520d628f None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:37.655869 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:37.656931 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:37.657531 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:37.657987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:37.668794 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:37.670242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52bc93-b3b8-4bb9-83ec-b99936a22ff8 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:37.682070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1392/2774] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:43.187065 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:INSERT=2,keystone:SELECT=87 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:47.233503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:47.303708 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf31ae1-d6a7-4d42-ad62-8c37def4cfa6 service swift] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:47.308932 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:47.309809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:47.310132 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:47.311022 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] PATH_INFO: `/v3/users/13a4b9f4052647ef8a45ba9fe6c2655d` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:47.312202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a4b9f4052647ef8a45ba9fe6c2655d)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:47.314182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:47.569263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a4b9f4052647ef8a45ba9fe6c2655d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:47.579312 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a4b9f4052647ef8a45ba9fe6c2655d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:47.605492 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a4b9f4052647ef8a45ba9fe6c2655d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:47.606724 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a4b9f4052647ef8a45ba9fe6c2655d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:47.607354 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a4b9f4052647ef8a45ba9fe6c2655d'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:47.676384 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=34 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:47.723619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1383/2775] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:47 2022] DELETE /identity/v3/users/13a4b9f4052647ef8a45ba9fe6c2655d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:47.728549 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:47.728951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:47.729305 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:47.750113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:47.752174 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:47.823580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1393/2776] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:47.836185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:47.838626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:47.935375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2751b8-b3be-40aa-aed5-bad497a16519 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:47.940968 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:47.942785 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:47.943239 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:47.943523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:47.954761 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:47.956208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:47.967328 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1384/2777] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:48.513990 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:48.515429 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:48.540018 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e34c7-87a4-4b4b-9549-2f7f207b8dce None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:48.557999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:48.559116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:48.559658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:48.560311 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] PATH_INFO: `/v3/projects/9b0ca3fc5cc94408b72516b5b5567152` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:48.562687 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0ca3fc5cc94408b72516b5b5567152)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:48.566848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:48.702995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0ca3fc5cc94408b72516b5b5567152', 'request_id': 'req-fcdfa9d0-21f4-4756-af23-bf76961be5ae'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:48.792335 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0ca3fc5cc94408b72516b5b5567152', 'request_id': 'req-fcdfa9d0-21f4-4756-af23-bf76961be5ae'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:48.799955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0ca3fc5cc94408b72516b5b5567152', 'request_id': 'req-fcdfa9d0-21f4-4756-af23-bf76961be5ae'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:48.814405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] The token cache is being invalidate because project 9b0ca3fc5cc94408b72516b5b5567152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:48.815026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:48.817211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:48.821663 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1394/2778] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:48 2022] DELETE /identity/v3/projects/9b0ca3fc5cc94408b72516b5b5567152 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:48.822664 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:48.828406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:48.835094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405c04d-f456-4039-bba1-03ecba642052 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:48.907093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:48.907757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:48.908035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:48.908307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] PATH_INFO: `/v3/users/b36c288049fa4034b1f420b17c5bb4f7` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:48.909471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36c288049fa4034b1f420b17c5bb4f7)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:48.912159 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:49.183514 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36c288049fa4034b1f420b17c5bb4f7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:49.190446 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36c288049fa4034b1f420b17c5bb4f7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:49.204117 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36c288049fa4034b1f420b17c5bb4f7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:49.204826 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36c288049fa4034b1f420b17c5bb4f7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:49.205481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-5290fa72-7196-458a-98cd-bf9978ef719a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36c288049fa4034b1f420b17c5bb4f7'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:49.297689 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1385/2779] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:48 2022] DELETE /identity/v3/users/b36c288049fa4034b1f420b17c5bb4f7 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:49.800454 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:49.803196 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:49.816637 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfa9d0-21f4-4756-af23-bf76961be5ae admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:49.880986 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:49.882543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:49.882829 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:49.883108 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] PATH_INFO: `/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:49.884278 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879dcb13f5ec40d68d17921ac8d8fceb)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:49.886208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.039339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:50.055693 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879dcb13f5ec40d68d17921ac8d8fceb', 'request_id': 'req-814086d7-87a1-47bc-a338-5968a2567760'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:50.186196 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.186698 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.186798 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.197600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879dcb13f5ec40d68d17921ac8d8fceb', 'request_id': 'req-814086d7-87a1-47bc-a338-5968a2567760'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:50.204909 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879dcb13f5ec40d68d17921ac8d8fceb', 'request_id': 'req-814086d7-87a1-47bc-a338-5968a2567760'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:50.206862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:50.208307 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:50.216619 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] The token cache is being invalidate because project 879dcb13f5ec40d68d17921ac8d8fceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:50.216939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:50.217585 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-814086d7-87a1-47bc-a338-5968a2567760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:50.219072 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1395/2780] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:49 2022] DELETE /identity/v3/projects/879dcb13f5ec40d68d17921ac8d8fceb => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:50.227325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de5eb-d6b2-420c-942d-af25b44e85ab None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.227626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de5eb-d6b2-420c-942d-af25b44e85ab None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.227887 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de5eb-d6b2-420c-942d-af25b44e85ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.248042 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-cb4de5eb-d6b2-420c-942d-af25b44e85ab None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:50.249286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-cb4de5eb-d6b2-420c-942d-af25b44e85ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:50.266718 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1386/2781] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:33:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:50.279405 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.280776 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:50.309658 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1396/2782] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:50.317793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.318147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.318478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.319554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:50.345055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:50.346293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:50.355492 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1397/2783] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:50.359547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd441875-7a02-42aa-b065-26e8b958e1bb None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.368850 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.369015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.369015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.369015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.369015 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.369406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9188c-e6f9-40b6-996d-b7ff1b09eb06 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.373813 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.374397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.374674 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.374950 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.375459 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.375570 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:50.375876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.377098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.377482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.385496 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1387/2784] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:33:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:33:50.389157 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1398/2785] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:33:50.393395 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.404994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde86b0-6bf8-4b99-b1c4-1ee75d939ae9 service swift] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.409766 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.420436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.420436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.420436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.420436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.420436 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.632597 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1388/2786] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:50 2022] POST /identity/v3/projects => generated 383 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:50.632745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.642681 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4338c-ed01-453b-9eec-852d1012c86e admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.699517 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.700122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.700405 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.700738 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.701398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.703318 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.810002 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1399/2787] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:50 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:50.813828 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.825104 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8444-44eb-4c71-aad1-bf5e0ed90843 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.830162 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.830780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.831061 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.831336 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.832469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.834318 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.844837 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1389/2788] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:50.849529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.859642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079930b0-0eef-4d6b-8c25-9f48cd03c130 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.864299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.864925 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.865207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.865482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.866125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.867718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.878814 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1400/2789] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:50.883001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.896133 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:50.898165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:50.898302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e986-069e-4ce7-bcb5-a7c22c940a20 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.903886 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.904648 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.905096 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.905435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] PATH_INFO: `/v3/projects/dc5ea7a1b6a54cd2af5d648f09aaed63/users/f5975379433a4e31a242764f58954f43/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.912824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a62ecb-20eb-4bae-ba85-cf01aa717273 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:50.917987 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:50.918849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:50.919107 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:50.919205 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] PATH_INFO: `/v3/users/3df08ebbcddd47caa9c68c7901534b10` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:50.920299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3df08ebbcddd47caa9c68c7901534b10)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.921958 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5ea7a1b6a54cd2af5d648f09aaed63, user_id=f5975379433a4e31a242764f58954f43, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:50.922248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.924566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da5f90-226c-4b48-9465-ce99dc7bc799 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:50.928410 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:51.027343 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1390/2790] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:50 2022] PUT /identity/v3/projects/dc5ea7a1b6a54cd2af5d648f09aaed63/users/f5975379433a4e31a242764f58954f43/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:51.032462 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:51.032771 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:51.033051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:51.051422 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:51.052658 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:51.118029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1391/2791] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:51.122602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:51.123942 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:51.150958 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3df08ebbcddd47caa9c68c7901534b10'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:51.158447 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3df08ebbcddd47caa9c68c7901534b10'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:51.173636 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3df08ebbcddd47caa9c68c7901534b10'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:51.174459 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3df08ebbcddd47caa9c68c7901534b10'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:51.175132 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3df08ebbcddd47caa9c68c7901534b10'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:51.190874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975bc77-8438-4fe8-92a5-7c458f1ee9c8 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:51.195026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:51.195630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:51.198449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:51.198449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:51.198449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:51.210728 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:51.212414 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b6ee8-2e66-4c90-b84a-4dda9a7a2235 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:51.222300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1392/2792] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:51.276217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1401/2793] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:50 2022] DELETE /identity/v3/users/3df08ebbcddd47caa9c68c7901534b10 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:51.280629 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:51.280957 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:51.281234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:51.288566 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:51.301503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:51.302949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:51.363757 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1393/2794] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:51.374505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:51.376234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:51.438046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c779af-3f13-4f6b-a784-a576b1389679 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:51.443480 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:51.445019 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:51.445299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:51.445569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:51.455696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:51.457434 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10c2f-3bc5-408e-980d-8fcac8f25758 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:51.467993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1402/2795] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:52.028481 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.029949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:52.043077 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a6108-6d67-453c-8cd7-008689fceba0 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.048283 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.048948 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.049223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.049501 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] PATH_INFO: `/v3/projects/38628e1a0d9b41dc9feb9bb102641ec2` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.050630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=38628e1a0d9b41dc9feb9bb102641ec2)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.053905 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.163898 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38628e1a0d9b41dc9feb9bb102641ec2', 'request_id': 'req-0486a7bd-7443-4047-b4c9-09107d4c21bd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:52.237494 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38628e1a0d9b41dc9feb9bb102641ec2', 'request_id': 'req-0486a7bd-7443-4047-b4c9-09107d4c21bd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:52.244806 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38628e1a0d9b41dc9feb9bb102641ec2', 'request_id': 'req-0486a7bd-7443-4047-b4c9-09107d4c21bd'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:52.257213 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] The token cache is being invalidate because project 38628e1a0d9b41dc9feb9bb102641ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:52.257531 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:52.258182 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-0486a7bd-7443-4047-b4c9-09107d4c21bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:52.259715 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1394/2796] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:52 2022] DELETE /identity/v3/projects/38628e1a0d9b41dc9feb9bb102641ec2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:52.264700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.265199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.265199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.284597 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:52.285998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:52.346293 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1403/2797] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:52.354397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.354679 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.354952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.375244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:52.379595 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:52.385665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1395/2798] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:52.390064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.400339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abd1fb-633b-4073-bbbd-62248f8bf433 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.405266 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.405881 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.408944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.408944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.408944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:52.408944 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.409357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.422154 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1404/2799] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 02:33:52.427197 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.442613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c116107-0993-43e6-b1b5-be883c96fe72 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.447644 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.448508 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.448821 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.449097 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.449848 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.451777 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.573309 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1396/2800] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:52 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:52.576359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.587451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322b7b5-0c61-454a-9422-7c5ebb0534a3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.600199 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:52.645743 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.651125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.651125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.651125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.651125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.651125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.793348 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1405/2801] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:52 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:52.797601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.810124 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebaf5a-5acb-4e25-8b34-5cea829717d2 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.815847 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.817469 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.817772 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.818095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.818951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.820639 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.834768 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1397/2802] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:52.838962 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.851746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8e97-69e0-454b-8ad3-431abca1a0cd admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.858649 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.859526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.863746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.863746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.863746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.863746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1956ce-dea6-445d-8b0e-051df1755640 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:52.878627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1406/2803] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:52.883618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:52.895782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af9431-6d9b-45ac-b5e1-6213ee2b5b9e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:52.900209 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:52.901028 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:52.901310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:52.901645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] PATH_INFO: `/v3/projects/a4d83d9d83074e05845589117643b62d/users/5f6fff82cccc4be2b45c725efbf8163b/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:52.920889 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d83d9d83074e05845589117643b62d, user_id=5f6fff82cccc4be2b45c725efbf8163b, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:52.924067 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:53.122984 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1398/2804] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:52 2022] PUT /identity/v3/projects/a4d83d9d83074e05845589117643b62d/users/5f6fff82cccc4be2b45c725efbf8163b/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:53.192828 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:53.193046 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:53.193302 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:53.221677 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] MFA Rules not processed for user `5f6fff82cccc4be2b45c725efbf8163b`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:53.223327 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:53.319575 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1407/2805] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:53.331873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:53.334844 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:53.366859 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:53.431319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61747d-9423-4353-9c22-c1b46ef7213d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:53.435829 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:53.436599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:53.436994 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:53.437346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:53.448232 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:53.449663 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b732a1-d9c7-4803-9355-0465b7faa3bc service cinder] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:53.461085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1399/2806] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:53.488172 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:53.566403 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910fa11-2564-47b0-86d8-b9e22a6cf851 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:53.574113 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:53.574774 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:53.575054 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:53.575324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] PATH_INFO: `/v3/users/5f6fff82cccc4be2b45c725efbf8163b` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:53.576504 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6fff82cccc4be2b45c725efbf8163b)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:53.578767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:53.713212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:53.872097 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6fff82cccc4be2b45c725efbf8163b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:53.880270 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6fff82cccc4be2b45c725efbf8163b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:53.895626 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6fff82cccc4be2b45c725efbf8163b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:53.896164 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6fff82cccc4be2b45c725efbf8163b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:53.897014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6fff82cccc4be2b45c725efbf8163b'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:53.978102 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1408/2807] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:53 2022] DELETE /identity/v3/users/5f6fff82cccc4be2b45c725efbf8163b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:53.979293 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:53.979604 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:53.979880 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:54.007884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:54.009509 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:54.091933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1400/2808] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:54.104026 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:54.105535 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:54.175523 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:54.177833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:54.195212 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ebd30-4747-44ed-a18a-ce5b3e4d39f4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:54.200156 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893cd6a-440b-4c7f-8310-3d458b82d3b9 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:54.200949 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:54.201812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:54.202155 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:54.202490 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:54.207412 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:54.213240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:54.213240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:54.213240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] PATH_INFO: `/v3/users/ffcf7b8ddd454d9fa3b32f86906cedd5` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:54.213240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffcf7b8ddd454d9fa3b32f86906cedd5)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:54.213240 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:54.214236 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:54.215811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:54.228694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1409/2809] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:54.497322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffcf7b8ddd454d9fa3b32f86906cedd5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:54.516515 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffcf7b8ddd454d9fa3b32f86906cedd5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:54.538098 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffcf7b8ddd454d9fa3b32f86906cedd5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:54.539177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffcf7b8ddd454d9fa3b32f86906cedd5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:54.540601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffcf7b8ddd454d9fa3b32f86906cedd5'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:54.715747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1401/2810] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:33:54 2022] DELETE /identity/v3/users/ffcf7b8ddd454d9fa3b32f86906cedd5 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:54.759898 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:54.761388 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:54.777997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29402caa-1153-4fec-b1ca-56ec15dd85de service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:54.895886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:54.896634 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:54.896941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:54.897225 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] PATH_INFO: `/v3/projects/a4d83d9d83074e05845589117643b62d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:54.900694 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d83d9d83074e05845589117643b62d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:54.902666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.090592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:55.091890 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d83d9d83074e05845589117643b62d', 'request_id': 'req-74be686b-d757-4130-b6b9-75e1967779d9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.112637 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.115349 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.134286 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4699-1071-4a26-937a-6dad9ff6ff52 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.158598 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:55.186248 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d83d9d83074e05845589117643b62d', 'request_id': 'req-74be686b-d757-4130-b6b9-75e1967779d9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.196717 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d83d9d83074e05845589117643b62d', 'request_id': 'req-74be686b-d757-4130-b6b9-75e1967779d9'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.211635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] The token cache is being invalidate because project a4d83d9d83074e05845589117643b62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:55.212119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.213158 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-74be686b-d757-4130-b6b9-75e1967779d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.216853 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1410/2811] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:54 2022] DELETE /identity/v3/projects/a4d83d9d83074e05845589117643b62d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:55.221162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55fd36-6a67-4954-90a9-0e232d2b27d5 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.221844 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55fd36-6a67-4954-90a9-0e232d2b27d5 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.223605 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55fd36-6a67-4954-90a9-0e232d2b27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.228465 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.230050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.230455 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.230748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] PATH_INFO: `/v3/projects/414b6a313558425cbd3ed5c6da5405a9` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.231997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414b6a313558425cbd3ed5c6da5405a9)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.234411 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.247031 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fa55fd36-6a67-4954-90a9-0e232d2b27d5 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:55.248314 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fa55fd36-6a67-4954-90a9-0e232d2b27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.259488 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1411/2812] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:55.267114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.267395 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.267672 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.291163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:55.292415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.301655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1412/2813] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:55.306725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.315827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a004-cb70-491a-8733-dcce146180d0 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.321213 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.321863 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.322147 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.322425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] PATH_INFO: `/v3/domains` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.323309 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80538) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:55.323613 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.325217 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.336739 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1413/2814] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:33:55.341060 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.350942 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c355e-c265-418a-848e-ed44ce08e73a admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.355733 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.356425 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.356732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.357012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] PATH_INFO: `/v3/projects` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.357635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.359528 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.379936 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414b6a313558425cbd3ed5c6da5405a9', 'request_id': 'req-8605998e-56af-4818-8be2-7d0b6a78e2c8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.506971 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414b6a313558425cbd3ed5c6da5405a9', 'request_id': 'req-8605998e-56af-4818-8be2-7d0b6a78e2c8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.511461 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414b6a313558425cbd3ed5c6da5405a9', 'request_id': 'req-8605998e-56af-4818-8be2-7d0b6a78e2c8'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.511588 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1414/2815] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:55.515291 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.526147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] The token cache is being invalidate because project 414b6a313558425cbd3ed5c6da5405a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:33:55.530095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.530095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-8605998e-56af-4818-8be2-7d0b6a78e2c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:33:55.530095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1402/2816] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:33:55 2022] DELETE /identity/v3/projects/414b6a313558425cbd3ed5c6da5405a9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:55.535874 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078f4f-43a7-4f9f-a1f4-2a637393ca73 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.537115 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078f4f-43a7-4f9f-a1f4-2a637393ca73 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.537504 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078f4f-43a7-4f9f-a1f4-2a637393ca73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.559447 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-86078f4f-43a7-4f9f-a1f4-2a637393ca73 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:55.560895 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-86078f4f-43a7-4f9f-a1f4-2a637393ca73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.584916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc5e31-edb4-4b31-888b-82821e77603f admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.596922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.598067 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.598526 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.598807 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.599533 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.601569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.638016 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1403/2817] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:55.647743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.648023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.648302 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.670472 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:55.671524 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.680846 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1404/2818] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:55.684955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.695246 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720abd1c-dc61-4985-8edf-5ceef8470702 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.701250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.701937 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.702287 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.702630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] PATH_INFO: `/v3/domains` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.703389 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 02:33:55.703818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.705518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.722397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1405/2819] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Aug 8 02:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 02:33:55.725418 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.734367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4e8-0604-484e-afe9-2e619c770042 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.740216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.740922 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.741202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.741485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.742102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.744068 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.766864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1415/2820] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:55.770365 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.773298 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:55.787569 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8458-c95e-4cee-b990-2ecb2570b200 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.792802 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.793415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.793767 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.794056 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.794702 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.796899 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.810796 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1416/2821] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:55.811924 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.823655 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb27eee-38c4-4fd5-b164-d984e57e9a17 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.829420 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.830251 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.830602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.830935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.831647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.834070 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.850521 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1417/2822] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:55.851593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.862793 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ae3b-7c5e-49c8-9637-b5a1e5d21af3 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.867608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.868494 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.868965 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.869329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] PATH_INFO: `/v3/projects/c30c9b3f01e740c2a329123736eb2e5e/users/0d75fc1643de49c680b5cb6b977fdb0c/roles/43be248131c345f4b8664cc353124638` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.891130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c9b3f01e740c2a329123736eb2e5e, user_id=0d75fc1643de49c680b5cb6b977fdb0c, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.895475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e3ef-3716-4f48-b927-55c3dfdba3f3 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:55.897014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1406/2823] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:55.897381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:55.979585 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f848-39d2-4ab8-b455-a6fa19b651ca admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:55.987372 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:55.988635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:55.988635 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:55.988907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] PATH_INFO: `/v3/users` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:55.989693 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:55.991714 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.053897 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1418/2824] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:55 2022] PUT /identity/v3/projects/c30c9b3f01e740c2a329123736eb2e5e/users/0d75fc1643de49c680b5cb6b977fdb0c/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:56.064088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.064609 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.065089 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.089960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1407/2825] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:55 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:56.098928 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.114385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] MFA Rules not processed for user `0d75fc1643de49c680b5cb6b977fdb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:56.119745 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.120907 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163c0b1-ec05-482c-a98b-34eb4f1c2da4 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.205516 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.206250 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.206525 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.206795 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.208375 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.210273 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.212283 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1419/2826] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:56.224148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.227161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1408/2827] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:56.227339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.228834 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.239498 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366bc8-ee03-4f6a-899f-d2a7276e7fe7 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.246108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.246108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.246108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.246108 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.246606 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.248300 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.262350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1409/2828] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:56.263914 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.275665 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4761-2b54-4504-8a2c-3f52712bf3fd admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.281267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.282042 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.282435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.282813 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] PATH_INFO: `/v3/projects/4e2c1907fad4499aae599d49e1d2bc80/users/b92e9b1d8c96434086055f58b119ed57/roles/43be248131c345f4b8664cc353124638` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.294948 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6614e1-d09b-4c7c-8930-d78ee6e1777e None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.301849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c925987dc66a445dae3cd8eee2f316ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.302589 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.302719 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.302994 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.306630 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2c1907fad4499aae599d49e1d2bc80, user_id=b92e9b1d8c96434086055f58b119ed57, role_id=43be248131c345f4b8664cc353124638)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.311024 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.312805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.314445 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7839fb-3ea1-4f01-aaed-3841c96cea09 service cinder] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.326692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1420/2829] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:33:56.472007 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1410/2830] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:56 2022] PUT /identity/v3/projects/4e2c1907fad4499aae599d49e1d2bc80/users/b92e9b1d8c96434086055f58b119ed57/roles/43be248131c345f4b8664cc353124638 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:33:56.477183 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038791-9008-4cc9-9f9d-67f0ccec79f7 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.477466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038791-9008-4cc9-9f9d-67f0ccec79f7 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.477737 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038791-9008-4cc9-9f9d-67f0ccec79f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.496543 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1d038791-9008-4cc9-9f9d-67f0ccec79f7 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:56.497776 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1d038791-9008-4cc9-9f9d-67f0ccec79f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.576378 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1421/2831] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:56.594267 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:56.601919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:56.673041 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:56.682226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26c0d9-5ef8-4e14-b201-6f00021b8f75 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:56.685803 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:56.686419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:56.686743 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:56.687054 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:56.698825 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:56.700863 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:56.718547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1411/2832] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:33:56.745095 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:57.688044 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:57.688544 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:57.688644 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:57.721654 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] MFA Rules not processed for user `f5975379433a4e31a242764f58954f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:57.723321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:57.810786 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1422/2833] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:33:57.814793 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:57.817569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:57.832726 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274649fd-d8f7-4665-a41c-fab6811f143a service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:57.837565 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:57.838184 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:57.838467 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:57.838742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:57.839500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:57.841577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:57.965599 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1412/2834] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:33:57.969635 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:57.983983 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491ee62-88bc-40ce-a23e-96f746237306 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.056732 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.057805 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.058032 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.058397 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.059252 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.061346 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fd5a1-773d-49a1-bc46-4048ba7f149f admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.166086 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1423/2835] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:33:57 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 02:33:58.170634 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.182150 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825248d-ff68-4212-be33-fe0324ed7c37 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.192386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.193076 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.193347 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.193482 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.194019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.194297 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.194569 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.195225 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.196974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.208955 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1413/2836] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:58.216346 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.219557 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] MFA Rules not processed for user `0d75fc1643de49c680b5cb6b977fdb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:33:58.227974 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b302bf8-b799-4559-bd43-75545a9fc395 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.233596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:33:58.235090 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.235890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.236003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.236308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] PATH_INFO: `/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040/users/ef12f9b1b7204464bff52fb1d3047451/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.253244 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c0ed8ba9c64f2591c5f2e9107f6040, user_id=ef12f9b1b7204464bff52fb1d3047451, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.256081 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.306812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1424/2837] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:33:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:33:58.383852 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1414/2838] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:58 2022] PUT /identity/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040/users/ef12f9b1b7204464bff52fb1d3047451/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:58.385324 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.400162 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833807be-b944-477f-986f-5e5cf8cdddfe None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.460416 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:33:58.483325 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.484708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.484708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.484708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.485538 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.487344 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2871e-be46-4649-a7f5-3d1f3512b172 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.503121 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1425/2839] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:33:58.505017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:33:58.519148 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0979f-ccc3-427c-b511-989d916b18ab admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:33:58.526092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:33:58.527023 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] REQUEST_METHOD: `PUT` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:33:58.527988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:33:58.528602 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] PATH_INFO: `/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040/users/ef12f9b1b7204464bff52fb1d3047451/roles/c555140ef854486a808b99fb021d4982` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:33:58.530721 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c0ed8ba9c64f2591c5f2e9107f6040, user_id=ef12f9b1b7204464bff52fb1d3047451, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:33:58.534192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:33:58.565009 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1415/2840] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:33:58 2022] PUT /identity/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040/users/ef12f9b1b7204464bff52fb1d3047451/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:33:58.601513 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:04.297091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:04.297091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:04.297091 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:04.374286 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] MFA Rules not processed for user `b92e9b1d8c96434086055f58b119ed57`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:04.375593 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:04.419088 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:04.581600 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1426/2841] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:04.586552 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:04.721633 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e5397-217a-4610-898d-de45da1f883c admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:04.732308 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:04.733884 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:04.734949 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:04.735263 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:04.736165 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:04.740420 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:04.849111 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1416/2842] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Aug 8 02:34:04 2022] POST /identity/v3/projects => generated 377 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:04.850755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:04.870222 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41c9f1-3a01-47ae-acbc-0b7fdfdbb42b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:04.959201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:04.959976 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:04.960296 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:04.960705 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] PATH_INFO: `/v3/users` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:04.961527 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:04.963582 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:05.075627 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1427/2843] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Aug 8 02:34:04 2022] POST /identity/v3/users => generated 295 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 02:34:05.077350 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:05.090784 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e25946-335a-4b31-9a61-88f7d48d6a69 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:05.095926 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:05.096445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:05.096755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:05.097535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:05.098439 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:05.100102 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:05.115979 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1417/2844] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:05.117148 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:05.128390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f181-c785-4402-9bbf-625e74043c6d admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:05.133258 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:05.133926 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:05.137603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:05.137603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] PATH_INFO: `/v3/projects/a2898509aa594b5194ac2282f285ce5d/users/aed17f99e0d94375a8bdb2156b738062/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:05.153888 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2898509aa594b5194ac2282f285ce5d, user_id=aed17f99e0d94375a8bdb2156b738062, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:05.156603 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:05.239474 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1428/2845] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:05 2022] PUT /identity/v3/projects/a2898509aa594b5194ac2282f285ce5d/users/aed17f99e0d94375a8bdb2156b738062/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:05.243315 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:05.260367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15172d09-f756-48b1-88f2-bc37401c89e8 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:05.347219 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:05.355893 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:05.369626 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:05.370794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] PATH_INFO: `/v3/roles` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:05.377189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:05.379657 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:05.387969 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:05.397915 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1418/2846] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Aug 8 02:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 02:34:05.405130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:05.423168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583199e7-c8ca-4759-a29d-0eefc3eae838 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:05.430875 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:05.432389 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:05.439550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:05.439550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] PATH_INFO: `/v3/projects/a2898509aa594b5194ac2282f285ce5d/users/aed17f99e0d94375a8bdb2156b738062/roles/c555140ef854486a808b99fb021d4982` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:05.439550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2898509aa594b5194ac2282f285ce5d, user_id=aed17f99e0d94375a8bdb2156b738062, role_id=c555140ef854486a808b99fb021d4982)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:05.447598 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58729-b635-4bbe-977e-6fc333861fb1 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:05.481124 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1429/2847] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Aug 8 02:34:05 2022] PUT /identity/v3/projects/a2898509aa594b5194ac2282f285ce5d/users/aed17f99e0d94375a8bdb2156b738062/roles/c555140ef854486a808b99fb021d4982 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:05.702934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:05.705110 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:05.841437 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03ba2-1e62-447e-add4-2a843435230d admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:05.960449 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:05.961517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:05.961517 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:05.961744 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:05.963600 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:06.072548 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:06.075649 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:06.091653 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1419/2848] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:08.664968 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656df32-f6af-4498-923a-e1140ecbac66 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:08.665518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656df32-f6af-4498-923a-e1140ecbac66 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:08.665518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656df32-f6af-4498-923a-e1140ecbac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:08.722261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5656df32-f6af-4498-923a-e1140ecbac66 None None] MFA Rules not processed for user `ef12f9b1b7204464bff52fb1d3047451`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:08.724400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5656df32-f6af-4498-923a-e1140ecbac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:08.856466 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:08.900340 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1430/2849] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:08.914591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:08.916130 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:09.018659 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ee301-a66f-401b-bcb8-5e2716d20bff service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:09.023331 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '144f1e9a377c4dd098112690cfdf24bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:09.024210 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:09.024619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:09.025026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:09.063092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:09.117341 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:09.118933 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:09.135157 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1420/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:09.549006 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:09.549226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:09.549501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:09.574631 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:09.576200 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:09.653256 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1431/2851] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:09.669849 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:09.676541 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:09.786259 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d999ea-f8f3-41cb-a5b5-62369b63320f service glance] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:09.792540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:09.793344 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:09.793723 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:09.794075 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:09.806114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:09.807960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d0900-ad9e-48da-bcb8-3902769494c3 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:09.821048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1421/2852] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:16.021385 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:16.033849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618a944-6fef-4d37-84ab-d02acbe66a21 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:16.044444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:16.045363 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:16.047596 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:16.048053 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:16.054386 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:16.259645 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:16.262150 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:16.279189 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1432/2853] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:17.197592 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718836-4d0e-46dd-b2bc-b63b01361379 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:17.198173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718836-4d0e-46dd-b2bc-b63b01361379 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:17.198173 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718836-4d0e-46dd-b2bc-b63b01361379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:17.282716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-4c718836-4d0e-46dd-b2bc-b63b01361379 None None] MFA Rules not processed for user `aed17f99e0d94375a8bdb2156b738062`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:17.285174 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4c718836-4d0e-46dd-b2bc-b63b01361379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:17.429017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1422/2854] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:17.446239 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:17.447703 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:17.598592 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735df7-bcb9-4190-aa2d-06a026aa7686 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:17.617077 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:17.617720 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:17.618007 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:17.618284 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:17.624907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=100 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:17.698817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:17.700704 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:17.724957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1433/2855] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:21.713256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419079e-9afb-4bee-b77c-4ead732cd276 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.713769 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419079e-9afb-4bee-b77c-4ead732cd276 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.713944 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419079e-9afb-4bee-b77c-4ead732cd276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.735266 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-b419079e-9afb-4bee-b77c-4ead732cd276 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:21.736601 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-b419079e-9afb-4bee-b77c-4ead732cd276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:21.745872 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1423/2856] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:21.766415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:21.782947 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42921f-f51d-44ca-93d1-fda1e5ce77a5 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:21.790281 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:21.791263 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:21.791885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:21.792380 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:21.803501 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:21.805505 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:21.815567 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1434/2857] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:23.946867 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9811-9cf9-47e9-a3a8-c928cb1f16cc None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:23.947352 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9811-9cf9-47e9-a3a8-c928cb1f16cc None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:23.947823 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c9811-9cf9-47e9-a3a8-c928cb1f16cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:23.989953 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-7a0c9811-9cf9-47e9-a3a8-c928cb1f16cc None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:23.994033 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7a0c9811-9cf9-47e9-a3a8-c928cb1f16cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:24.007444 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1424/2858] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:34:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:24.026424 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:24.040886 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b667ac-c710-4a53-ac0a-5af24f2a82b2 service swift] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:24.050512 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:24.055475 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:24.060810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:24.061414 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:24.080170 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:24.085432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:24.101264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1435/2859] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:34:25.140419 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e7b-63ec-49f3-94c2-5ce7c50b1cd2 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:25.141039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e7b-63ec-49f3-94c2-5ce7c50b1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:25.141039 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8e7b-63ec-49f3-94c2-5ce7c50b1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:25.186572 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e22c8e7b-63ec-49f3-94c2-5ce7c50b1cd2 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:25.187870 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e22c8e7b-63ec-49f3-94c2-5ce7c50b1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:25.208537 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1425/2860] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:25.230518 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:25.243014 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a45be1-20e2-4cdf-a656-1eb94a77efdf service swift] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:25.249242 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:25.250119 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:25.250676 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:25.251114 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:25.261955 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:25.265529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def12162-5500-4fb5-bbda-8e6d6d1f71fd service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:25.276602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1436/2861] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:27.850613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.851092 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.851189 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.871421 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.871936 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.872071 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.876453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:27.877778 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.896190 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] MFA Rules not processed for user `f5975379433a4e31a242764f58954f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:27.897876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:27.905164 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1426/2862] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:27.906238 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:27.908472 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1437/2863] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:27.923409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207c241-be8e-421c-b6d7-f653eafe4fee None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:27.930530 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:27.935161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:27.935161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:27.935161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:27.951221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:27.953001 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:27.958651 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=100 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:27.963699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1427/2864] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:34:28.290264 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:28.291752 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:28.307708 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d71793-4a35-4908-ab69-3733b2f36e9a None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:28.312711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:28.313368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:28.313642 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:28.313908 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] PATH_INFO: `/v3/users/8be4cb4f1c594f42825eb40985a29cac` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:28.315028 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be4cb4f1c594f42825eb40985a29cac)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:28.317299 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:28.591590 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be4cb4f1c594f42825eb40985a29cac'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.603343 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be4cb4f1c594f42825eb40985a29cac'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.622432 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be4cb4f1c594f42825eb40985a29cac'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.625064 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be4cb4f1c594f42825eb40985a29cac'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.625772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be4cb4f1c594f42825eb40985a29cac'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:28.755218 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1438/2865] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:28 2022] DELETE /identity/v3/users/8be4cb4f1c594f42825eb40985a29cac => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:29.063089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.088767 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7dbd5-0659-47ed-837e-8a7ca9d593c0 service swift] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.160920 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.161234 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.161518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.161794 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] PATH_INFO: `/v3/projects/fa4edbfca7b749ff9be3268fa6b81264` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.163041 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4edbfca7b749ff9be3268fa6b81264)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.164982 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.325488 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4edbfca7b749ff9be3268fa6b81264', 'request_id': 'req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.430977 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4edbfca7b749ff9be3268fa6b81264', 'request_id': 'req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.437249 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4edbfca7b749ff9be3268fa6b81264', 'request_id': 'req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.455094 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] The token cache is being invalidate because project fa4edbfca7b749ff9be3268fa6b81264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:29.455326 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.455973 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.458454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1428/2866] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:29 2022] DELETE /identity/v3/projects/fa4edbfca7b749ff9be3268fa6b81264 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:29.462740 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:29.465629 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:29.480448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd83df-e8a9-4175-a45d-96589c55b1e4 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:29.518359 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:29.556838 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:29.557478 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:29.557810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:29.558069 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] PATH_INFO: `/v3/users/a2690d27ae8e4125a9640e9bf45e202d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:29.562940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2690d27ae8e4125a9640e9bf45e202d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:29.562940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:29.872632 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2690d27ae8e4125a9640e9bf45e202d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.880833 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2690d27ae8e4125a9640e9bf45e202d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.892911 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2690d27ae8e4125a9640e9bf45e202d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.893647 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2690d27ae8e4125a9640e9bf45e202d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.894450 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2690d27ae8e4125a9640e9bf45e202d'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:29.968078 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1439/2867] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:29 2022] DELETE /identity/v3/users/a2690d27ae8e4125a9640e9bf45e202d => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:30.267535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:30.269017 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:30.282227 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1fa0d-96a5-44c3-98e9-8a4ba41af03e admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:30.351369 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:30.355961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:30.355961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:30.355961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] PATH_INFO: `/v3/projects/7fc76c9217b04e3ebdc10482112ff802` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:30.355961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc76c9217b04e3ebdc10482112ff802)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:30.355961 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:30.501453 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:30.514645 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc76c9217b04e3ebdc10482112ff802', 'request_id': 'req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.597759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc76c9217b04e3ebdc10482112ff802', 'request_id': 'req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.610603 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc76c9217b04e3ebdc10482112ff802', 'request_id': 'req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.623186 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] The token cache is being invalidate because project 7fc76c9217b04e3ebdc10482112ff802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:30.623482 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.624228 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-e2ee5f10-e5dc-43ee-a1ee-fb31c095ef73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:30.625759 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1429/2868] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:30 2022] DELETE /identity/v3/projects/7fc76c9217b04e3ebdc10482112ff802 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 02:34:31.022448 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.023940 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.042473 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a78e2-dc75-4c66-82ae-f16114aba2da admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.102370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.103207 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.103487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.103755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] PATH_INFO: `/v3/users/0d75fc1643de49c680b5cb6b977fdb0c` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.104907 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d75fc1643de49c680b5cb6b977fdb0c)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.106826 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.326903 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d75fc1643de49c680b5cb6b977fdb0c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.332684 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:31.334864 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d75fc1643de49c680b5cb6b977fdb0c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.346812 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d75fc1643de49c680b5cb6b977fdb0c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.348249 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d75fc1643de49c680b5cb6b977fdb0c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.349047 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d75fc1643de49c680b5cb6b977fdb0c'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:31.484138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1440/2869] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:34:31 2022] DELETE /identity/v3/users/0d75fc1643de49c680b5cb6b977fdb0c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:31.488554 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.489503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.489503 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.511579 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] MFA Rules not processed for user `979e7eb5d5594f0ea4d2839dfb577133`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:31.513026 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.575177 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1430/2870] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:31.585757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.587247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.650352 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e5f0-224d-4d19-93c1-149e3557dcd6 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.654177 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.654775 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.655051 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.655321 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.665831 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.677406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fc481-82b4-469e-b26a-3b1c485845b2 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:31.678292 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1441/2871] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:34:31.904003 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:31.905518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:31.922199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45eb41a-e17a-438a-80ba-d502e2f4da23 None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:31.929988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:31.929988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:31.929988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:31.929988 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] PATH_INFO: `/v3/projects/c30c9b3f01e740c2a329123736eb2e5e` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:31.930441 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30c9b3f01e740c2a329123736eb2e5e)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:31.932596 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.103076 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30c9b3f01e740c2a329123736eb2e5e', 'request_id': 'req-9bc9388a-f691-432a-8c1f-af336bd02107'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.178401 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30c9b3f01e740c2a329123736eb2e5e', 'request_id': 'req-9bc9388a-f691-432a-8c1f-af336bd02107'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.185192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30c9b3f01e740c2a329123736eb2e5e', 'request_id': 'req-9bc9388a-f691-432a-8c1f-af336bd02107'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.200295 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] The token cache is being invalidate because project c30c9b3f01e740c2a329123736eb2e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:34:32.200573 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.201406 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:34:32.202890 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1431/2872] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:34:31 2022] DELETE /identity/v3/projects/c30c9b3f01e740c2a329123736eb2e5e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:34:32.647772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4602e0-647f-4479-aaa0-1afea25a2576 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.648247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4602e0-647f-4479-aaa0-1afea25a2576 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.648342 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4602e0-647f-4479-aaa0-1afea25a2576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.669670 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-5b4602e0-647f-4479-aaa0-1afea25a2576 None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:32.669670 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-5b4602e0-647f-4479-aaa0-1afea25a2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.726972 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1442/2873] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Aug 8 02:34:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 02:34:32.740891 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:32.742409 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:32.798435 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:32.831591 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9388a-f691-432a-8c1f-af336bd02107 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:32.840370 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:32.841051 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:32.841332 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:32.853923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:32.853923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:32.853923 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:32.864380 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1432/2874] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:34:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 02:34:40.556206 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:40.556873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:40.556873 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:40.578811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] MFA Rules not processed for user `b92e9b1d8c96434086055f58b119ed57`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:34:40.578811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:40.633666 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1443/2875] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:34:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:34:43.117038 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=12 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:50.626730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=85,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:34:59.541005 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:34:59.542426 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:59.602187 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11a6f6-9b90-4eef-b3db-d7555704a1d6 service swift] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:34:59.605908 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:34:59.606540 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:34:59.606818 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:34:59.607089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:34:59.608820 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:34:59.672011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:34:59.673574 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:34:59.683780 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1433/2876] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:34:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:00.814083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:00.815497 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.828246 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbc76-2428-44b7-95fc-b33edfb1be70 None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:00.831808 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:00.832412 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:00.832718 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:00.832995 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:00.834692 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:00.900294 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:00.901824 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:00.911500 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1444/2877] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 02:35:09.679716 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=50 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:10.907138 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=30 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:24.268223 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:24.277618 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b28d-aec3-42be-8fbf-7fec38b66f71 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:24.281319 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:24.284029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:24.284029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:24.284029 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:24.284325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:24.360800 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:24.362361 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:24.373950 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1434/2878] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:25.566073 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:35:25.575153 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9218f-e7cb-4612-af8e-988c66849a52 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:35:25.578772 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:35:25.579398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:35:25.579678 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:35:25.579957 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:35:25.581711 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:35:25.595746 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:35:25.597220 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:35:25.609401 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1445/2879] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:35:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:35:34.368594 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=34 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:35:35.603304 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:09.950443 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:09.960202 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665cf34-226c-4b7f-bddc-162c39b23878 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:09.963954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:09.964567 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:09.964938 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:09.965221 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:09.966896 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:09.978050 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:09.979809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668e247-c078-47b8-813d-6d6554040826 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:09.991123 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1435/2880] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:16.602667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:16.604055 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:16.617867 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1be17-3b26-429e-8339-d5b8c345ffcc service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:16.623406 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:16.623999 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:16.624277 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:16.624550 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:16.626226 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:16.638667 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:16.640166 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:16.651529 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1446/2881] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:18.001543 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] REQUEST_METHOD: `POST` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:18.002019 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:18.002125 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:18.049882 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.auth.core [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] MFA Rules not processed for user `76ca42843b2941e0860bca74f5d7c5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:36:18.057952 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:18.069035 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1436/2882] 10.222.0.27 () {58 vars in 967 bytes} [Mon Aug 8 02:36:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:36:18.110394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:18.132951 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8e2bf-6aac-4617-b5cf-f16d566d5de6 service nova] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:18.138247 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '171d2d63e59b41a783cdbc61db788122', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:18.138848 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:18.139125 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:18.139394 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:18.147966 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:18.149747 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:18.157800 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1447/2883] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Aug 8 02:36:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 02:36:22.603192 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:22.605248 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:22.677114 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730b525-572e-42a7-b565-0b6f81f62bbf None None] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:22.682006 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:22.682615 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:22.682919 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:22.683195 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:22.684934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:22.696801 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:22.698367 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:22.709454 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1437/2884] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:36:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:36:26.373211 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:26.374599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:26.445444 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15b4a1-3038-4213-a363-48fb2f1caaf1 service swift] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:26.450201 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:26.451116 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:26.451396 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:26.451669 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] PATH_INFO: `/v3/users/f5975379433a4e31a242764f58954f43` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:26.452817 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5975379433a4e31a242764f58954f43)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:26.454763 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:26.609468 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5975379433a4e31a242764f58954f43'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.616769 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5975379433a4e31a242764f58954f43'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.628329 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5975379433a4e31a242764f58954f43'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.629005 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5975379433a4e31a242764f58954f43'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.629602 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5975379433a4e31a242764f58954f43'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:26.726339 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1448/2885] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:26 2022] DELETE /identity/v3/users/f5975379433a4e31a242764f58954f43 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:26.979809 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:26.991216 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e74b01-84d1-483e-be44-42ca6ccc9808 service placement] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:27.050381 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:27.051014 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:27.051290 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:27.051680 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] PATH_INFO: `/v3/projects/dc5ea7a1b6a54cd2af5d648f09aaed63` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:27.053468 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5ea7a1b6a54cd2af5d648f09aaed63)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:27.055547 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:27.188252 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:27.195321 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'request_id': 'req-08985920-3270-4813-9f55-60c9d6b2b542'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.291865 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'request_id': 'req-08985920-3270-4813-9f55-60c9d6b2b542'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.297960 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5ea7a1b6a54cd2af5d648f09aaed63', 'request_id': 'req-08985920-3270-4813-9f55-60c9d6b2b542'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.310093 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] The token cache is being invalidate because project dc5ea7a1b6a54cd2af5d648f09aaed63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:27.310383 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.311020 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.312495 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1438/2886] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:26 2022] DELETE /identity/v3/projects/dc5ea7a1b6a54cd2af5d648f09aaed63 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:27.316730 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:27.318519 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:27.331783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2400084-5b87-4417-a07b-3ad7f017be54 admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:27.373700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:27.388039 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:27.388688 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:27.388977 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:27.389250 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] PATH_INFO: `/v3/users/ef12f9b1b7204464bff52fb1d3047451` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:27.390370 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef12f9b1b7204464bff52fb1d3047451)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:27.392315 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:27.586599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef12f9b1b7204464bff52fb1d3047451'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.594048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef12f9b1b7204464bff52fb1d3047451'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.607048 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef12f9b1b7204464bff52fb1d3047451'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.607751 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef12f9b1b7204464bff52fb1d3047451'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.608347 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-42a01bf0-137a-4207-ab4e-46244ba6f733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef12f9b1b7204464bff52fb1d3047451'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:27.691811 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1449/2887] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:36:27 2022] DELETE /identity/v3/users/ef12f9b1b7204464bff52fb1d3047451 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:27.961918 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:36:27.963432 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:36:27.976954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08985920-3270-4813-9f55-60c9d6b2b542 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:36:28.037544 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:36:28.038284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:36:28.038485 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:36:28.038786 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] PATH_INFO: `/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:36:28.039985 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c0ed8ba9c64f2591c5f2e9107f6040)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:36:28.041954 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:36:28.179843 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c0ed8ba9c64f2591c5f2e9107f6040', 'request_id': 'req-4f60bb16-d195-46ae-af91-422105524abe'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.223152 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c0ed8ba9c64f2591c5f2e9107f6040', 'request_id': 'req-4f60bb16-d195-46ae-af91-422105524abe'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.229902 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c0ed8ba9c64f2591c5f2e9107f6040', 'request_id': 'req-4f60bb16-d195-46ae-af91-422105524abe'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.241959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] The token cache is being invalidate because project e7c0ed8ba9c64f2591c5f2e9107f6040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:36:28.242284 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.242959 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:36:28.244747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1439/2888] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:36:27 2022] DELETE /identity/v3/projects/e7c0ed8ba9c64f2591c5f2e9107f6040 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:36:37.687938 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:36:38.238139 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:02.570415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:02.570415 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:02.571799 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:02.590917 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] MFA Rules not processed for user `b92e9b1d8c96434086055f58b119ed57`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:37:02.592260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:02.650122 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1450/2889] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:37:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:37:02.960833 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:02.962245 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:03.018191 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60bb16-d195-46ae-af91-422105524abe admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:03.022557 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:03.022924 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:03.023199 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:03.023471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:03.025303 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:03.037181 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:03.038610 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:03.047208 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1440/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:04.790168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:04.791563 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:04.862608 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e49ee3-7ff8-4f54-87e4-1accaa30a66b None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:04.867761 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:04.868368 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:04.868803 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:04.869082 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:04.870827 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:04.883012 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:04.884660 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:04.896186 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1451/2891] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:13.043031 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=26 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:14.892083 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=57 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:23.465814 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:23.469011 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:23.545306 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a1d13-b47b-43ea-bb66-d456f30d08e5 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:23.551382 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:23.552479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:23.552479 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:23.552747 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:23.554440 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:23.567151 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:23.568748 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:23.583584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1441/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:27.897580 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:37:27.911036 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81d8eb-6e9b-41a9-97b7-b4e42f8fe7be service placement] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:37:27.916561 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:37:27.917208 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:37:27.917487 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:37:27.917762 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:37:27.919502 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:37:27.931998 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:37:27.933625 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08d514-e7da-43c5-84a1-bc891a79a885 service nova] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:37:27.945009 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1452/2893] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:37:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:37:33.574976 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=34 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:37:37.940773 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:38:32.912619 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:38:32.914046 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:32.927862 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9eaaaf-d2df-4611-8daa-57b2161c2635 service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:38:32.933365 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:38:32.933951 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:38:32.934261 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:38:32.934535 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:38:32.936185 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:38:32.947873 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:38:32.949475 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:38:32.959487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1442/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:38:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:38:42.955256 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:03.418851 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] REQUEST_METHOD: `POST` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:03.419916 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:03.420127 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:03.439846 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.auth.core [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] MFA Rules not processed for user `b92e9b1d8c96434086055f58b119ed57`. Rule list: `[]` (Enabled: `True`). {{(pid=80538) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 02:39:03.441130 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:03.448354 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1453/2895] 10.222.0.27 () {58 vars in 958 bytes} [Mon Aug 8 02:39:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 02:39:13.434382 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=3 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:39:17.257445 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:39:17.270584 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7214f2-5717-4f57-8dba-e23d8cb02ea9 service placement] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:39:17.276161 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfff7acf53f4d779549a9688117f05d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:39:17.276782 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:39:17.277089 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:39:17.277364 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:39:17.279085 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:39:17.290539 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:39:17.292118 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:39:17.301226 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1443/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:39:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:39:27.296964 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:40:53.803409 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:53.814261 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df43cd-3fd9-4dba-83b5-c21dda927fba None None] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:53.818050 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:53.818715 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:53.819058 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:53.819390 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:53.822357 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:53.836126 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:53.838227 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:53.849461 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1454/2897] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:40:55.036037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:40:55.045987 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a382-3379-4385-b82b-f1df1eff07bf service nova] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:40:55.049583 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:40:55.050207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:40:55.050487 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:40:55.050779 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:40:55.052471 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:40:55.066500 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:40:55.067912 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:40:55.079325 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1444/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:40:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:03.844426 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=6 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:05.074188 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=6 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:06.933700 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:06.935599 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:06.946323 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43f388-c7af-427a-ac70-fad5bdf2f354 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:06.950856 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ce21b04744a41b590dbc8a8640ffc3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:06.951451 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:06.951725 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:06.951993 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:06.953939 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:07.017398 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:07.018935 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:07.029224 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1455/2899] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:41:14.041207 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.042824 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.058506 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768c2fc-c7f7-4d42-9e4d-ddadb1a9c069 service neutron] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.064106 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.064765 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.065048 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.065322 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] PATH_INFO: `/v3/users/b92e9b1d8c96434086055f58b119ed57` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.066578 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b92e9b1d8c96434086055f58b119ed57)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.068577 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.213147 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b92e9b1d8c96434086055f58b119ed57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.220423 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b92e9b1d8c96434086055f58b119ed57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.232532 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b92e9b1d8c96434086055f58b119ed57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.233243 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b92e9b1d8c96434086055f58b119ed57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.233850 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b92e9b1d8c96434086055f58b119ed57'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.278614 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1445/2900] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:14 2022] DELETE /identity/v3/users/b92e9b1d8c96434086055f58b119ed57 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:14.530757 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.541810 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c62914-3718-4f4b-a134-9c358dd0f2f7 service neutron] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.599849 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.600483 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.600795 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.601075 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] PATH_INFO: `/v3/projects/4e2c1907fad4499aae599d49e1d2bc80` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.602233 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2c1907fad4499aae599d49e1d2bc80)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.604165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:14.703400 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2c1907fad4499aae599d49e1d2bc80', 'request_id': 'req-fb18bff5-c991-4981-8ab2-298820f6c0df'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.753922 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2c1907fad4499aae599d49e1d2bc80', 'request_id': 'req-fb18bff5-c991-4981-8ab2-298820f6c0df'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.759595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2c1907fad4499aae599d49e1d2bc80', 'request_id': 'req-fb18bff5-c991-4981-8ab2-298820f6c0df'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.771536 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] The token cache is being invalidate because project 4e2c1907fad4499aae599d49e1d2bc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:14.771821 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.772499 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:14.773997 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1456/2901] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:14 2022] DELETE /identity/v3/projects/4e2c1907fad4499aae599d49e1d2bc80 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:14.777997 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:14.779536 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:14.792699 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9f2fa-a064-406e-8acb-dbdc567a8273 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:14.848396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:14.849062 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:14.849340 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:14.849613 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] PATH_INFO: `/v3/users/aed17f99e0d94375a8bdb2156b738062` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:14.850742 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed17f99e0d94375a8bdb2156b738062)` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:14.852654 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:15.023278 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed17f99e0d94375a8bdb2156b738062'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.031934 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed17f99e0d94375a8bdb2156b738062'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.045576 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed17f99e0d94375a8bdb2156b738062'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.046178 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed17f99e0d94375a8bdb2156b738062'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.046695 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.notifications [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed17f99e0d94375a8bdb2156b738062'} {{(pid=80537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.048396 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:15.133943 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1446/2902] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Aug 8 02:41:14 2022] DELETE /identity/v3/users/aed17f99e0d94375a8bdb2156b738062 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:15.359234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Authenticating user token {{(pid=80538) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:41:15.360885 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.fernet_utils [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80538) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:41:15.378260 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18bff5-c991-4981-8ab2-298820f6c0df admin admin] Validating token access rules against request {{(pid=80538) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:41:15.405411 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:15.445595 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e7eb5d5594f0ea4d2839dfb577133', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f307f793c6c464eafa52c1ca17d9bc6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80538) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:41:15.446234 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:41:15.446513 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:41:15.446782 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] PATH_INFO: `/v3/projects/a2898509aa594b5194ac2282f285ce5d` {{(pid=80538) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:41:15.447941 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2898509aa594b5194ac2282f285ce5d)` {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:41:15.449876 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] RBAC: Authorization granted {{(pid=80538) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:41:15.603696 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2898509aa594b5194ac2282f285ce5d', 'request_id': 'req-c2734044-c6ee-4c05-9de2-613aa53700ec'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.659783 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2898509aa594b5194ac2282f285ce5d', 'request_id': 'req-c2734044-c6ee-4c05-9de2-613aa53700ec'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.665484 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2898509aa594b5194ac2282f285ce5d', 'request_id': 'req-c2734044-c6ee-4c05-9de2-613aa53700ec'} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.677165 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] The token cache is being invalidate because project a2898509aa594b5194ac2282f285ce5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80538) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 02:41:15.677476 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.678163 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG keystone.notifications [None req-c2734044-c6ee-4c05-9de2-613aa53700ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80538) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 02:41:15.679755 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: [pid: 80538|app: 0|req: 1457/2903] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Aug 8 02:41:15 2022] DELETE /identity/v3/projects/a2898509aa594b5194ac2282f285ce5d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 02:41:25.202735 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:41:25.674168 dv-cinder-740146-18-iscsi devstack@keystone.service[80538]: DEBUG dbcounter [-] [80538] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80538) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 02:43:40.864792 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Authenticating user token {{(pid=80537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 02:43:40.867276 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.005397 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f29a4c-2a9f-4ce4-b25d-166d32005d06 admin admin] Validating token access rules against request {{(pid=80537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 02:43:41.064121 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c46c81aeb14e189c21aa484404ad9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d51bb7c64ea44d0a65391702c0820ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 02:43:41.064755 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] REQUEST_METHOD: `GET` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 02:43:41.065037 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] SCRIPT_NAME: `/identity` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 02:43:41.065310 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 02:43:41.067036 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.fernet_utils [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 02:43:41.078099 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 02:43:41.079662 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f2606-9ec9-41b1-bb66-e2c53096bcbd service placement] RBAC: Authorization granted {{(pid=80537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 02:43:41.089764 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: [pid: 80537|app: 0|req: 1447/2904] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Aug 8 02:43:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 02:43:51.085518 dv-cinder-740146-18-iscsi devstack@keystone.service[80537]: DEBUG dbcounter [-] [80537] Writing DB stats keystone:SELECT=89 {{(pid=80537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}